HAProxy Kubernetes Ingress Controller 1.10 is now the latest version. Learn more
We’re proud to announce the release of version 1.8 of the HAProxy Kubernetes Ingress Controller!
In this release, we added support for full rootless mode, Prometheus metrics for the controller itself, and examples that are synchronized with our Helm chart. In this blog post, you will learn more about the changes in this version.
Register for our webinar "Using CRDs to Improve Quality of Life in Kubernetes" to learn more about this release.
Running Unprivileged
Prior to this version, the ingress controller process ran with elevated privileges (root) inside its Docker container. The reason why we had to default to root privileges was to be able to bind to privileged ports (those below 1024), such as the standard HTTP and HTTPS ports 80 and 443. However, containers that run with elevated privileges pose a risk, since it becomes easier to escape the container sandbox and access the host system. In fact, some hosted Kubernetes providers such as Google Kubernetes Engine (GKE) Autopilot even disallow running privileged containers altogether.
Workarounds exist, including having a sidecar container running sysctl net.ipv4.ipunprivilegedport_start=0 next to the ingress controller container. However, GKE Autopilot would not allow that either since it requires a privileged container to run sysctl.
Now, we no longer run a privileged container by default. Instead, we drop privilege to UID 1000 and GID 1000 for all processes in the container, including the S6, ingress controller, and HAProxy processes. We grant the HAProxy binary the CAPNETBIND_SERVICE capability via the setcap command when building the ingress controller Docker image so that it can bind to the desired ports:
| setcap cap_net_bind_service=+ep haproxy |
This is possibly a breaking change for older host systems such as Debian Stretch, Ubuntu Willy, and Ubuntu Xenial, because their kernels disable some needed features for advanced multi-layered unification filesystem (AUFS), which had been the preferred storage driver for Docker 18.06 and older. Most Docker / Kubernetes installations have already moved to the OverlayFS or OverlayFS2 storage driver, making this problem mostly a legacy systems issue.
HAProxy Kubernetes Ingress Controller 1.8 permits running unprivileged, and the latest Helm chart uses that mode by default.
Diagnostic pprof Data
The ingress controller exposes an endpoint for viewing pprof diagnostic data.
You enable it with the controller argument --pprof. With version 1.8, an additional argument, --controller-port, lets you change the endpoint’s port. By default, if pprof is enabled, it listens at port 6060, which is compatible with version 1.7. This port is also shared with the new Prometheus metrics endpoint, which is described in the next section.
To enable this feature when using Helm, pass the controller.extraArgs parameter:
| $ helm install kubernetes-ingress haproxytech/kubernetes-ingress \ | |
| --set "controller.extraArgs={--pprof,--controller-port=6060}" |
Access the pprof diagnostics page at the path /debug/pprof.
Prometheus Metrics
This version adds a new endpoint for collecting Prometheus metrics that are specific to the inner workings of the controller, such as the amount of memory allocated and CPU time spent.
Metrics for the load balancing portion of the controller have been available since version 1.0, which you can access on port 1024 at the URL path /metrics, but this version includes metrics for the controller itself. Access the new metrics on port 6060 at the URL path /metrics.
In order to enable them, two new controller arguments were introduced:
--controller-port– port where this will be exposed (defaults to 6060)--prometheus– to enable prometheus
Here is an example, using Helm:
| $ helm install kubernetes-ingress haproxytech/kubernetes-ingress \ | |
| --set "controller.extraArgs={--prometheus,--controller-port=6060}" |
Default Backend
The ingress controller routes traffic to a configured set of pods by evaluating routing rules defined in Ingress objects. However, if no ingress rule matches, the request will go to a default backend, which displays a 404 error message. Until now, the --default-backend-service controller argument pointed to the pod that ran the default backend application. However, that approach required running yet another service in your cluster.
With this release, another option is available. If you omit --default-backend-service then a default service controller will create one that’s part of the controller itself. If you run the ingress controller in external mode, you can set --default-backend-port to define a port where the default backend service runs. If you don’t set this, port 6061 will be used.
Examples
The project’s GitHub repository now contains a collection of examples. The examples are synced and produce the same setup as when you use Helm (same namespaces, service names, etc.), which is especially useful for learning how to create everything by hand with kubectl and then comparing it with Helm’s setup. A shortcut for initializing the examples is to invoke the command make example.
Additional Improvements
Several other improvements join this release:
You can restrict access to your applications by setting the blacklist or whitelist annotations. These now accept pattern files that store the IP addresses, which makes it easier to support long lists of IP addresses.
Support for the new annotation
client-strict-snihas been added. It returns a TLS error if no certificate is found for the client SNI.Support for a default ingress class has been added.
Contributions
We’d like to thank the code contributors who helped make this version possible!
Moemen MHEDHBI | TEST OPTIM REORG BUILD FEATURE BUG DOC |
Ivan Matmati | REORG TEST BUG FEATURE |
Dinko Korunic | FEATURE BUILD |
Zlatko Bratkovic | BUG CLEANUP FEATURE DOC BUILD |
Nico Braun | FEATURE |
Davor Kapsa | DOC |
Petr Studeny | BUG |
Jonas Weber | FEATURE |
Frank Villaro-Dixon | BUILD |
Daniel Lenar | BUG |
Jakub Granieczny | BUG |
LarsBingBong | BUG |
