HAProxy Kubernetes Ingress Controller 1.11 is now available. For our enterprise customers, HAProxy Enterprise Kubernetes Ingress Controller 1.11 is coming soon and will incorporate the same features. In this release, we enhanced security through the adoption of rootless containers, graduated our custom resource definitions to v1, made them easier to manage, and introduced support for the QUIC protocol.
Additionally, we've simplified version compatibility with HAProxy and included a reload/restart module to log and manage configuration changes better. These advancements are designed to provide a more secure, efficient, and user-friendly platform for managing ingress traffic in Kubernetes environments. In this blog post, you will learn more about HAProxy Kubernetes Ingress Controller 1.11 changes.
Version compatibility with HAProxy
We're simplifying how we version our Ingress Controller to make it easier to understand and keep up with future updates. HAProxy Kubernetes Ingress Controller 1.11 uses HAProxy 2.8, but this will be the last time that the two version numbers are different. Starting with our next release, the version number of the Ingress Controller will match the version of HAProxy it uses. The next version number will be 3.0, and it will match up with HAProxy 3.0. This update eliminates confusion regarding the association between the Ingress Controller and HAProxy versions.
Custom Resource Definitions (CRDs) v1 (Backend, Defaults, and Global)
In HAProxy Kubernetes Ingress Controller 1.11, we've upgraded our Custom Resource Definitions (CRDs) to v1. This upgrade brings full support for all HAProxy configuration options.
Additionally, we've updated the group of definitions to ingress.v1.haproxy.org. While putting v1 into the name of the CRD group might seem unconventional, it's a strategic choice designed to accommodate future versions. Kubernetes architecture is structured to avoid breaking changes in CRDs, as they are stored collectively. This approach works well for operators, where typically only one version runs in a cluster. However, for an ingress controller that might run multiple versions simultaneously, it's crucial to ensure that newer versions (such as a potential version 2) maintain existing configurations.
In environments with multiple teams, it's common for updates not to happen simultaneously across the board. Since custom resources ultimately interact with a unified cluster API, supporting multiple versions (even those with breaking changes) is essential and must not cause issues.
Using webhooks is not an applicable solution in this context, as the challenge lies in managing versions with breaking changes within the Kubernetes API itself. This update ensures that the Ingress Controller can evolve without impacting the broader cluster operation, maintaining stability and compatibility across different team deployments.
Avoid problems early with v1
A significant improvement from the alpha versions in the latest update is adopting a common expression language in version 1. This enhancement allows for thorough validation of configurations before they are inserted into Kubernetes, such as during a kubectl apply command.
This preemptive validation step helps avoid configurations that disrupt the controller by catching errors before the data is even saved in the Kubernetes API. This update brings the Ingress Controller's behavior in line with the HAProxy Data Plane API, which relies on the same definitions for operation. This ensures a more stable and reliable setup by catching errors early in the configuration process.
Transitioning from Alpha versions
In 1.11, the v1alpha1 and v1alpha2 CRD versions are now deprecated. While they remain supported for now, it's important to note that upcoming releases, starting with version 3.0, may no longer support these versions. If you are currently using v1alpha2 (noting that v1alpha1 was already deprecated in version 1.10), you will need to make some changes.
Specifically, you'll need to update the group for your Custom Resource Definitions (Backend, Defaults, and Global) and adjust your Role-Based Access Control (RBAC) rules to accommodate this new group. This step is crucial for ensuring your configurations remain compatible with future versions of our software.
Enhanced CRD management
.png)
In HAProxy Kubernetes Ingress Controller 1.11, we have introduced a simpler way to install and update our Custom Resource Definitions (CRDs). A new command-line option, --job-check-crd, has been added, enabling users to install or update definitions easily. This enhancement, which uses a Kubernetes Job to perform the update, ensures a smoother operation for managing CRDs and has been backported to all maintained versions for broader support.
For Helm users (available at Helm charts), this improvement is applied automatically.
A Kubernetes Job is a construct that allows executing one or more pods in your cluster, ensuring that a specified number of them terminate successfully. As opposed to regular pods that are designed to run indefinitely, Jobs are used to execute batch operations that run to completion. This makes Jobs ideal for tasks that need to be run only once or on a finite basis, such as data processing tasks, batch computations, or database migrations. When a Job is initiated, it starts pods that run the tasks defined in the Job's specifications. If a pod fails or is deleted due to a node failure, the Job controller will start a new pod in its place unless the Job is configured not to retry. You can set Jobs to run multiple pods in parallel, allowing for more efficient completion of the task at hand.
QUIC (Quick UDP Internet Connections) support
With version 1.11, we are introducing support for QUIC, a transport layer network protocol, which will be enabled automatically for users using certificates and TLS.
In instances where binding to the UDP port is not feasible or desired, you can turn off this feature simply by using the --disable-quic option.
Additionally, the options --quic-bind-port and --quic-announce-port allow you to tailor the QUIC protocol's port settings to your specific requirements. With --quic-bind-port, you can designate the precise port for QUIC binding. While this level of customization may only be necessary for some, it offers essential flexibility in environments with stricter policies where UDP may not be enabled in the same way as TCP.
To ensure clients can transition from HTTP/2 (or HTTP) to QUIC, controllers must announce which port the client can connect to, and the --quic-announce-port option allows you to do just that. Additionally, the --quic-alt-svc-max-age option lets you define how long the connection has to upgrade. These features aim to simplify your network management, enhancing performance and security with QUIC.
Advancing security with rootless containers
The S6 Overlay, our chosen init system, has been updated to the latest version, v3. This update is part of our ongoing commitment to enhance the security and functionality of the Ingress Controller container image, making it fully rootless and ensuring it operates seamlessly in read-only environments.
With this update, as a consequence of going fully rootless, we have changed the default ports for HTTP and HTTPS from the standard 80 and 443 to 8080 and 8443, respectively. This internal modification is designed not to disrupt existing setups, as the binding of external ports is contingent upon your specific configurations and the nature of your deployment. This adjustment is particularly noteworthy for those utilizing the controller in external mode, where it is employed directly as an application rather than as a Docker image.
To revert to the original port settings, you can use the https-bind-port and http-bind-port arguments to specify your preferred bind ports.
Additional changes are also needed when defining a Deployment. The security context must have:
runAsNonRootset totrue,allowPrivilegeEscalationset tofalse,seccompProfileneeds to be defined and set toRuntimeDefaulttype.
For Helm users, these changes are applied automatically.
Github Container (GHCR) Images
Docker images for our Ingress Controller are available on Docker Hub, where they've proudly surpassed 50 million downloads! Additionally, we started offering images on GitHub too. Those who wish to test the latest developments can also use nightly images.
You can find these packages at our HAProxy Technologies Kubernetes Ingress GitHub repository.
To pull an image, simply use the command docker pull ghcr.io/haproxytech/kubernetes-ingress:1.11.0.
Our nightly images are intended for development purposes and may exhibit behavior changes as we refine features leading up to the official release.
Controller port
We're making it easier for users to leverage pprof and Prometheus with our controller. To access these tools, you'll need to use the controller-port, which by default is set to 6060.
While this isn't a change in how things operate, we're clarifying the process to ensure you know exactly where to find these resources.
Upgraded logging for clearer insights and better control
We've introduced a reload/restart module to the Kubernetes Ingress Controller, enhancing transparency around configuration changes. This addition makes it easier to understand when, how, and why configuration changes were made throughout the controller's lifecycle, offering greater insight into the system's operation.
Logging in HAProxy Kubernetes Ingress Controller 1.11 includes unique transaction IDs in all log messages, where implementation is possible. This enhancement aids in accurately matching any failed transactions to their causes, whether due to incorrect settings or conditions. This makes it easier to find and understand the root cause of any problem by offering more precise information for troubleshooting and reporting.
These updates offer more clarity and control of your system's performance and security.
Notable additions
Disabling configuration snippets
The disable-config-snippets option allows you to turn off configuration snippets. This option accepts a comma-separated list, with possible values including backend, frontend, global, and all. You can combine these options in any way you need, with all conveniently disabling all snippets.
Config snippet validation
While we recommend using custom resource definitions, we understand the necessity and convenience of config snippets for specific scenarios. Since config snippets are integrated into the configuration precisely as they are provided, they require careful consideration to ensure they do not disrupt the overall configuration. To address this, we've enhanced the controller's resilience to errors in snippets, allowing for a more forgiving and robust handling of configurations. This improvement aims to provide you with peace of mind, knowing that minor mistakes won't compromise your entire system.
log-format-tcp
The log-format-tcp command sets the log format string for TCP traffic. It only applies to the TCP configmap specified by the command line option --configmap-tcp-services.
allow-list and deny-list
The terms whitelist and blacklist have been deprecated in favor of allow-list and deny-list, respectively. It's important to note that while these terms are still operational, we plan to phase them out in the future. We encourage you to start using allow-list and deny-list in your configurations to ensure a smooth transition when the older terms are eventually removed.
standalone-backend
The standalone-backend annotation has been introduced, enabling each ingress object to create a separate backend. While this approach may not be the standard practice, it offers enhanced customization for specific paths.
Documentation
You can find HAProxy documentation here. For those looking for our Ingress Controller documentation, it's readily available on the HAProxy's Kubernetes Ingress documentation page, as well as on the official GitHub repository.
These resources are designed to support you with detailed information and guides, ensuring a smooth experience with HAProxy and the HAProxy Kubernetes Ingress Controller.
Contributions
We’d like to thank the code contributors who helped make this version possible!
Contributor | Area |
Hélène Durand | BUG, CLEANUP, BUILD, DOC, FEATURE, TEST |
Ivan Matmati | FEATURE, DOC, BUG |
Vincent Gramer | BUG, DOC, FEATURE, OPTIM |
Dinko Korunic | BUG, BUILD, FEATURE |
Dario Tranchitella | TEST |
Fabiano Parente | BUG |
Alexis Vachette | BUG |
Conrad Hoffmann | BUG |
Michal Zielonka | BUG |
Zlatko Bratkovic | BUG, BUILD, CLEANUP, DOC, FEATURE, TEST |
Conclusion
HAProxy Kubernetes Ingress Controller 1.11 represents our commitment to delivering a secure, efficient, and user-friendly platform for managing ingress traffic. By embracing rootless containers, enhancing Custom Resource Definitions (CRDs) management as we graduate our CRDs to v1, and introducing QUIC protocol support, we are setting new standards for security and performance in Kubernetes solutions. Introducing a reload/restart module ensures users can manage configuration changes with greater clarity and control.
Looking ahead, we're focused on offering features that maximize HAProxy's benefits within Kubernetes, aiming for an even more powerful, scalable, and secure application delivery system.
Subscribe to our blog. Get the latest release updates, tutorials, and deep-dives from HAProxy experts.
