HAProxy Kubernetes Ingress Controller 3.0 is now available. For our enterprise customers, HAProxy Enterprise Kubernetes Ingress Controller 3.0 will arrive later this year and incorporate these same features. In this release, we've added TCP custom resource definitions (CRDs) to improve mapping, structuring, and validation for TCP services within HAProxy Kubernetes Ingress Controller. Say goodbye to messy service list management and "hello" to greater flexibility with HAProxy options for your K8s services.
In this blog post, we'll share a quick note on updated naming conventions before diving deeper into HAProxy Kubernetes Ingress Controller's headlining features.
Version compatibility with HAProxy
HAProxy Kubernetes Ingress Controller 3.0 is built with HAProxy version 3.0 and has now jumped from version 1.11 to version 3.0. Starting with this release, Kubernetes Ingress Controller's version number will match the version of HAProxy it uses. We hope this clarifies the link between HAProxy Kubernetes Ingress Controller and its baseline version of HAProxy, moving forward.
Custom Resource Definitions: TCP
Until now, mapping for TCP services was available through a custom ConfigMap using the --configmap-tcp-services flag. While this worked as expected, there were a few limitations we needed to address.
For example, ConfigMap alone doesn't have a standardized structure nor validation. Therefore, keeping a larger list of services tidy can be challenging. Additionally, only some HAProxy options (such as service, port, and SSL/TLS offloading) were available for those types of services.
The tcps.ingress.v1.haproxy.org definition, conversely, lets us define and use more HAProxy options than we could with ConfigMap.
Installing and getting to know TCP CRDs
If you're using Helm, the TCP services definition will be installed automatically. Otherwise, it's available as a raw YAML file via GitHub.
TCP custom resources (CRs) are namespaced and you can deploy several of them in a shared namespace. HAProxy will apply them all.
A TCP CR contains a list of TCP service definitions. Each service definition has:
A name
A frontend section containing two permitted components:
Any setting from client-native frontend model
A list of binds coupled with any settings from client-native bind models
A service definition that's a Kubernetes upstream Service/Port (the K8s Service and the deployed TCP CR must be in the same namespace).
Here's a simple example of a TCP service:
| apiVersion: ingress.v1.haproxy.org/v1 | |
| kind: TCP | |
| metadata: | |
| name: tcp-1 | |
| spec: | |
| - name: tcp-http-echo-8443 | |
| frontend: | |
| name: http-echo-445 | |
| tcplog: true | |
| binds: | |
| - name: mytcpapp | |
| port: 20000 | |
| service: | |
| name: http-echo | |
| port: 8443 |
How do we configure service and backend options? You can use the Backend Custom Resource (and reference it in the Ingress Controller ConfigMap, Ingress, or the Service) in conjunction with the TCP CR.
Mitigating TCP collisions
TCP services are tricky since they allow for unwanted naming and configuration duplications. This overlap can cause transmission delays and other performance degradations while impacting reliability.
Luckily, HAProxy can detect and manage two types of collisions:
Collisions on frontend names
Collisions on bind addresses and ports
If several TCP services across all namespaces encounter these collisions, HAProxy will only apply the one that was created first based on the older CreationTimestamp of the custom resource. This will generate a message in the log.
SSL/TLS in a TCP custom resource
Here's a quick example of a TCP service with SSL/TLS enabled:
| apiVersion: ingress.v1.haproxy.org/v1 | |
| kind: TCP | |
| metadata: | |
| name: tcp-1 | |
| spec: | |
| - name: tcp-http-echo-8443 | |
| frontend: | |
| name: fe-http-echo-8443 | |
| tcplog: true | |
| log_format: "%{+Q}o %t %s" | |
| binds: | |
| - name: v4 | |
| ssl: true | |
| ssl_certificate: tcp-test-cert | |
| port: 2000 | |
| - name: v4v6 | |
| address: "::" | |
| port: 2000 | |
| v4v6: true | |
| service: | |
| name: "http-echo" | |
| port: 8443 |
Keep in mind that ssl_certificate can be the following:
The name of a Kubernetes Secret (in the same namespace as the TCP CR) containing the certificate and key
A folder or filename on the pod's local filesystem, which was mounted as a Secret Volume
For example, you can mount an SSL/TLS Secret in the Ingress Controller Pod on a volume and reference the volume mount path in ssl_certificate. Without changing the Pod (or deployment manifest), you can instead use a Secret name within the ssl_certificate configuration. As a result, the certificate and key will be written in the Pod's filesystem at the etc/haproxy/certs/tcp path.
The TCP ConfigMap and TCP Custom Resources aren't compatible. If you use both (a TCP CR and the TCP ConfigMap with a TCP service on the same address/Port), this would lead to random configuration. Please ensure you deploy TCP Custom Resources and your TCP ConfigMap services using unique addresses and ports.
Custom Resource Definitions (CRDs): transitioning from alpha versions
In HAProxy Kubernetes Ingress Controller 1.11, we deprecated the v1alpha1 and v1alpha2 CRD versions. Only v1alpha2 is supported within version 3.0. However, this will be the last Kubernetes Ingress Controller release where this specific version is available. If you're currently using v1alpha2, we strongly recommend upgrading to the v1 version.
Breaking changes
If you're using --configmap-tcp-services, this release changes the default backend configuration for a TCP Service defined via annotation in your ConfigMap. Previously, any backend options defined in the ConfigMap (such as maxconn or server-slots) didn't apply to TCP backends. These options now apply to TCP backends defined via annotation in ConfigMap.
Contributions
HAProxy Kubernetes Ingress Controller's development thrives on community feedback and feature input. We’d like to thank the code contributors who helped make this version possible!
Contributor | Area |
Hélène Durand | FEATURE, BUG, TEST |
Ivan Matmati | FEATURE, BUG |
Dinko Korunić | FEATURE |
Olivier Doucet | FEATURE |
Fabiano Parente | BUG |
Petr Studeny | BUG |
jaraics | BUG |
Ali Afsharzadeh | BUILD |
Zlatko Bratković | BUILD, FEATURE, DOC, TEST |
Conclusion
HAProxy Kubernetes Ingress Controller 3.0 represents our commitment to delivering a flexible and efficient platform for managing ingress traffic. By extending our prior CRD support to include TCP CRDs, our Kubernetes solutions can meet even more use cases with less complexity.
To learn more about HAProxy Kubernetes Ingress Controller, follow our blog and browse our Ingress Controller documentation. If you want to see how HAProxy Technologies also provides external load balancing and multi-cluster routing alongside our ingress controller, check out our Kubernetes solutions and our K8s webinar.
Subscribe to our blog. Get the latest release updates, tutorials, and deep-dives from HAProxy experts.
