Announcing HAProxy Data Plane API 2.8

We are proud to announce that we have released HAProxy Data Plane API 2.8, available on our GitHub page

This release follows the recent HAProxy 2.8 release and incorporates its changes, along with some improvements and changes specific to the API. 

HAProxy Data Plane API 2.8 adds new keywords focused on QUIC, OCSP stapling, and tuning options that allow you to customize your HAProxy process using the HTTP REST API programmatically. This release also extends the Consul service discovery feature with options to customize which services you want to include in discovery, allow better control of their defaults, and integrate with Consul health checks.

Due to some licensing conflicts and for simplicity, we also removed the HCL configuration format for configuring the Data Plane API. From version 2.8 onwards, Data Plane API supports only the YAML format.

Extended HAProxy Configuration Keywords Support

Following the latest release of HAProxy, we are extending the configuration keyword support to support many new features in the load balancer.

The /services/haproxy/configuration/binds endpoint gained the following options:

client_sigalgs

Signing algorithms, each separated by a colon, dictate the algorithms a client can use in the CertificateVerify message during authentication.

no_alpn

HAProxy 2.8 sets an ALPN value automatically. Setting this to true turns that off.

quic-cc-algo

Sets the congestion control algorithm for QUIC listeners. It can be either cubic (the default) or newreno.

sigalgs

Signing algorithms, each separated by a colon, that HAProxy can use when signing the ephemeral public key during the TLS handshake.

thread

Defines the threads on which a listener can run. It accepts a comma-separated list of thread ranges.

The /services/haproxy/configuration/global endpoint gained the following options:

close_spread_time

Defines a span of time over which to spread the closing of idle connections, rather than closing them all at once. This option accepts an integer, in milliseconds.

cluster_secret

Defines an ASCII string secret used to derive stateless reset tokens for QUIC connections. When not set, the value is random.

httpclient_resolvers_disabled

Turns off the DNS resolvers section used by the Lua HTTP client. HAProxy generates this resolvers section automatically, which parses the /etc/resolv.conf file for DNS nameservers.You can set this to either enabled or disabled.

httpclient_resolvers_id

Sets the name of the resolvers section the Lua HTTP client should use. By default, it tries the resolvers section named default, which is hidden but can be turned off.

httpclient_resolvers_prefer

Changes the preferred address version. By default, the Lua HTTP client prefers IPv6 addresses from DNS records. You can set this to either ipv4 or ipv6. 

httpclient_ssl_ca_file

Sets the path to a CA file the Lua HTTP client uses to verify the server certificate of the HTTP service it connects to when that service is hosted over HTTPS. This might be necessary if the server certificate is not already trusted because its chained CA is unknown.

httpclient_ssl_verify

Disables verification of the server certificate of the service the Lua HTTP client is connecting to. This option accepts a value of none or required.

no-quic

Globally turns off the QUIC transport protocol. Any configured QUIC listeners will not bind to their addresses. Set this to true or false.

prealloc-fd

Opens the maximum file descriptor, which pre-allocates the kernel's data structures. Set this to true or false.

ssl_default_bind_client_sigalgs

Sets the global default for the signing algorithms clients can use in their CertificateVerify messages during authentication. Separate each algorithm by a colon.

ssl_default_bind_sigalgs

Sets the global default for the signing algorithms HAProxy can use when signing the ephemeral public key during the TLS handshake. Separate each algorithm by a colon.

h2_be_initial_window_size

Sets the initial window size for outgoing HTTP/2 connections. Set this to a number of bytes.

h2_be_max_concurrent_streams

Sets the maximum number of concurrent streams per outgoing HTTP/2 connection.

h2_fe_initial_window_size

Sets the initial window size for incoming HTTP/2 connections. Set this to a number of bytes.

h2_fe_max_concurrent_streams

Sets the maximum number of concurrent streams per incoming HTTP/2 connection. It applies to outgoing connections too, when h2_be_max_concurrent_streams is not set.

listener_default_shards

Sets the strategy by which a listener is sharded (a shortcut for defining multiple listeners for the same address). It accepts a value of by-process, by-thread, or by-group.

lua_burst_timeout

Sets a timeout for how long a Lua script can execute without yielding control back to the HAProxy process. Other timeouts exist for the total execution time of a script across yields. Set this to a number of milliseconds.

memory_hot_size

Sets the amount of memory to keep hot in the local cache for each thread. The default is 512 KB. It accepts a number of bytes.

quic_frontend_conn_tc_buffers_limit

Sets the maximum number of buffers allocated for a QUIC connection. The default is 30.

quic_frontend_max_idle_timeout

Sets a timeout for idle QUIC connections in milliseconds for frontends. The default is 30 seconds.

quic_frontend_max_streams_bidi

Sets the initial maximum number of bidirectional streams the remote peer will be authorized to open, which determines the number of concurrent client requests. The default is 100.

quic_max_frame_loss

QUIC is transported over the UDP protocol, which ordinarily would not guarantee message delivery. But QUIC does have loss detection and will retransmit lost frames. This setting counts the number of retransmissions and closes the connection if the limit has been reached. The default is 10.

quic_retry_threshold

Sets a threshold on the number of half-opened QUIC connections, which, if reached, dynamically enables the Retry feature. The default is 100.

quic_socket_owner

Sets the strategy by which QUIC connections use sockets to send and receive messages. Set this to either connection (the default) to allocate a socket per connection or listener to share a socket.

ssl_ocsp_update_max_delay

Sets the maximum interval in seconds between two automatic updates of the same OCSP response. It defaults to 1 hour.

ssl_ocsp_update_min_delay

Sets the minimum interval in seconds between two automatic updates of the same OCSP response. It defaults to 5 minutes.

stick_counters

Changes the number of stick table counters available. It defaults to 3.

The /services/haproxy/configuration/http_request_rules endpoint gained the following options for its type argument:

lua

Creates an http-request lua directive. Use this in combination with the lua_action and lua_params arguments.

sc-add-gpc

Creates an http-request sc-add-gpc directive.

sc-inc-gpc

Creates an http-request sc-inc-gpc directive.

The /services/haproxy/configuration/http_after_response_rules endpoint gained the following options for its type argument:

del-acl

Creates an http-after-response del-acl directive.

del-map

Creates an http-after-response del-map directive.

sc-add-gpc

Creates an http-after-response sc-add-gpc directive.

sc-inc-gpc

Creates an http-after-response sc-inc-gpc directive.

sc-inc-gpc0

Creates an http-after-response sc-inc-gpc0 directive.

sc-inc-gpc1

Creates an http-after-response sc-inc-gpc1 directive.

sc-set-gpt0

Creates an http-after-response sc-set-gpt0 directive.

The /services/haproxy/configuration/tcp_request_rules endpoint gained the following options for its action argument:

sc-add-gpc

Creates an sc-add-gpc action for a tcp-request connection, tcp-request content, or tcp-request session directive depending on how you set the type argument.

sc-inc-gpc

Creates an sc-inc-gpc action for a tcp-request connection, tcp-request content, or tcp-request session directive, depending on how you set the type argument.

The /services/haproxy/configuration/tcp_response_rules endpoint also gained the above actions.

Extend Consul Service Discovery Options

extend-consul-service-discovery-options-in-haproxy-data-plane-api-2-8

If you are using Consul to discover your services, you can use the Data Plane API to dynamically configure HAProxy from changes in Consul's services registry. You can use the /service_discovery/consul endpoint to set up parameters for the Data Plane API to connect to the Consul API and read Consul service discovery information. Read our blog post Consul Service Discovery for HAProxy, for an introduction to this topic.

In HAProxy Data Plane API 2.8, we’ve extended that feature with several new options to enable greater flexibility with your Consul integration.

We’ve added the defaults field, with which you can select a specific defaults section by its name from which the generated backends that represent Consul services will inherit settings. This option allows you to have different defaults sections for Consul service discovery, separated from other load balanced applications, for more flexibility in setting their other options.

In the snippet below of a configuration generated by the API, the backend section has inherited settings from the defaults section named consul:

defaults consul
  timeout connect 5s
  timeout client 30s
  timeout server 10s

//autogenerated consul backend from consul
backend consul-backend-127.0.0.1-80-test from consul
  server …
  …

Also, we’ve added two new fields that consider Consul health checks: health_check_policy and the health_check_policy_min. These fields allow you to define the health check conditions for each node in service for that node to be considered valid and configured as a backend server in the HAProxy configuration. You can define four different health check policies:

none

All the nodes for a service reported by Consul API are considered valid and will be configured. This is the default behavior.

any

A node is considered valid if any of the configured health checks is reporting passing on the Consul API.

all

A node is considered valid if all of the configured health checks are reporting passing on the Consul API.

min

A node is considered valid if the number of passing health checks is greater or equal to the health_check_policy_min configured value. If fewer health checks are configured, then the configured health_check_policy_min node will always be considered invalid.

In addition to those changes, we’ve added service_name_regexp to allow for more programmability in our Consul service discovery feature. This regular expression filters the reported services from the Consul API by their name. Only the services that match the given regular expression will be considered and configured in the HAProxy configuration file.

Breaking Change: Removing HCL Configuration Format

You can configure HAProxy Data Plane API using its configuration file. We support HCL and YAML file formats, so you can choose either. Due to the licensing conflict with libraries used to parse the HCL file format, we decided to stick with only one file format: YAML. It’s widely used and easy to read and write manually. So, for simplicity, we have removed the support for the HCL configuration file.

To ease the migration for users currently using the HCL configuration file format, we published a tool that converts a Data Plane API configuration file from HCL  to YAML. You can find the HCL-to-YAML conversion tool here.

After unpacking the binary from the archive, you can do the following:

hcl-converter dataplaneapi.hcl dataplaneapi.yml

On successful conversion, it will print out the following output:

Input file: dataplaneapi.hcl
Output file: dataplaneapi.yaml

Library Updates and Bug Fixes

In addition to the other changes, HAProxy Data Plane API 2.8 introduces stability and infrastructure improvements and bug fixes.

One of the bigger updates is that the Data Plane API project has been migrated to Go 1.20 along with the underlying libraries, config-parser, and client-native. This migration allows us to improve our codebase with all the new features, optimizations, and security fixes of Go 1.20.

In addition, we scanned all external dependencies and updated Data Plane API’s external libraries for bug and security fixes.

There have been 56 commits of bug fixes in the Data Plane API project and its underlying libraries since the v2.7.0 release.

Conclusion & Contributors

Contributor

Area

Andreas Mahnke

BUG

Dario Tranchitella

BUG | FEATURE | TEST

Dinko Korunic

OPTIMIZATION

George Vine

BUG | BUILD | FEATURE

Goran Galinec

BUILD | FEATURE

Jerermy Montupet

FEATURE

Marko Juraga

BUG | BUILD | FEATURE | TEST

Olivier Duclos

BUG | BUILD | DOCUMENTATION | FEATURE

Pratik Mohanty

BUG | FEATURE

Robert Maticevic

BUG | BUILD | FEATURE

Zlatko Bratkovic

BUILD

To check the latest release, visit our GitHub releases page.

Subscribe to our blog. Get the latest release updates, tutorials, and deep-dives from HAProxy experts.