Announcing HAProxy Data Plane API 3.0

Watch our on-demand webinar to explore the exciting new features of our HAProxy Data Plane API 3.0 release!

HAProxy Data Plane API 3.0 is now available! The latest version is hosted on our GitHub releases page.

This release follows the recent HAProxy 3.0 release and incorporates its changes, along with some improvements and changes specific to the API.

HAProxy Data Plane 3.0 adds multiple breaking changes. We'll cover the impacts of these changes in detail to highlight how your implementation and usage of Data Plane API may be affected. Despite some initial growing pains, v3 provides a stronger foundation for future development as we continue to modernize and simplify the API to meet users' needs. We recommend reading carefully through this upcoming section. 

This release also makes key improvements to the API, which we'll also expand upon later: 

• Users can now fetch a parent resource with all children embedded. 

• We've added a PUT operation on index-based resources. 

• Timeouts are now handled with your preferred time suffix. 

• The API's state data is removed from the configuration file. User-defined settings are now the configuration's sole focus. 

• Support has been added for every new keyword introduced in HAProxy 3.0. 

But that only scratches the surface. We're excited to show you the core functionality updates we've made in v3 and why they matter.

Breaking changes

This version introduces breaking changes described in this section.

Removed _version and data wrapper fields in responses

Each time the Data Plane API changes the load balancer configuration it increments the version of the file. This enables optimistic locking of the configuration to prevent a user from unintentionally overwriting another user's changes. In the initial design, each API endpoint returned JSON data with _version and data fields as top keys in the response object. Later, we also added a Configuration-Version HTTP header, making the configuration version available in both the wrapped response and the header. 

In v3, we've removed the redundant _version and data fields from the response and moved the JSON previously under data to the top—making it easier to parse. This simplifies the process of changing an existing resource. Users can leverage this flat structure to fetch the resource, change a few fields, and send back the same object without needing to unpack it from the data field.

Renamed the defaults resource

In the early stages of the Data Plane API, we intended to generate and manage only one defaults section within the HAProxy configuration by using the /v2/services/haproxy/configuration/defaults endpoint. This decision to manage just one was based on the fact that support for assigning defaults sections names—by which you could indicate which to inherit settings from—didn't originally exist. Assigning a name to a defaults section, although valid syntactically, had no functional meaning in HAProxy.

However, HAProxy 2.4 introduced keywords to the frontend and backend sections, giving meaning to these names and allowing users to create multiple, named defaults that frontends and backends could then inherit settings from. Since the defaults resource initially returned an object instead of a list, we introduced a new resource called named_defaults to maintain backward compatibility. This resource returned a list of defaults resources that names and could be referenced in backends and frontends sections. We kept the original defaults resource for backward compatibility as an object.

With v3 of the API, we're migrating named_defaults to simply defaults, breaking the old defaults behavior. The defaults resource now returns a list versus an object.

Moved child resources as nested resources

In v2, some resources represent lines within a section of HAProxy configuration, effectively making them child resources of the parent section resource. For example, the acl resource can be a child of a frontend or backend, and a server resource can be a child of a backend parent. 

You could previously fetch or update these resources in the API by identifying them with their index, which was the case for the acl resource. You could do the same by name for other resources, such as the servers resource. You'd then specify their parents using the parent_type and parent_name query-string parameters.

This process was unfortunately confusing, so we've redesigned child resources as nested resources in the URL. Because a child resource cannot simultaneously exist in multiple parent resources, it makes sense to include this in the parent resource's URL. This also boosts compatibility with many external tools. For example, you can build role-based access control (RBAC) rules more easily atop the new URL hierarchy without relying on query-string parameters.

Here's how the two implementations compare:

• In v2, you'd see /services/haproxy/configuration/acls?parent_type=frontend&parent_name=my-example-frontend

• In v3, you'll instead see /services/haproxy/configuration/frontends/my-example-frontend/acls

Removed explicit index field on index-based child resources

In v2 of the API, all index-based resources—which represent configuration lines that can be repeated within a section and are distinguished by the order of appearance—had an index field indicating the position of that specific resource. For example, acl resources get an index to indicate their order within the configuration.

We've removed that index field in v3, as we found it redundant and complicated to maintain when positions change. You can infer the ordering from the actual position in the returned list.

Removed support for HAProxy in multi-process mode

HAProxy previously had a feature that allowed it to run in multi-process mode. This mode has since been deprecated and removed following the introduction of multi-thread mode. In v2 of the API, we supported all keywords related to multi-process mode. Additionally, we supported multiple runtime APIs (one for each process mode), so all /runtime resources worked with multiple process support.

We've removed this support altogether in v3 alongside the following fields from the respective sections:

• Backend resource: bind_process

• Bind resource: process

• Defaults resource: bind_process

• Frontend resource: bind_process

• Global resource: nbproc

In addition to removing these fields, /runtime endpoints have changed.

The stats resource at /v2/services/haproxy/stats/native used to return an array of arrays. The top-level array held per-process stats, with each array element containing an array of stat lines for each object (server, backend, frontend). Now that multi-process mode support is ending, /v3/services/haproxy/stats/native returns an object with a single stats array for the server, backend, or frontend object you've requested. Similarly, we removed one layer of arrays that returned process information on a per-process basis. This change affected /v2/services/haproxy/runtime/stick_table such that the process field is removed from responses that previously indicated from which process the stick table information was being fetched.

In addition to this, the query string parameter process was removed from both /v2/services/haproxy/runtime/stick_tables and /v2/services/haproxy/runtime/stick_table_entries resources.

Reorganized global resource fields

One thing we set out to do with v3 was reorganize the global resource fields. The global resource is our largest if you measure by the number of fields, and it primarily had a flat structure. To improve readability of this resource on the API, we reorganized the fields into nested objects.

While making this change, we tried to adhere to HAProxy's field naming conventions and followed documentation hints when deciding how best to structure the revised fields. We tried to maintain only one level of nested objects to avoid unnecessary complexity. Some fields remain in the root level of the object, and some options are split into debug_options and performance_options as specified in the Configuration Manual.

We've also added an ssl_options nested object to cover all SSL directives. Other objects like tune_ssl_options, tune_vars_options, and tune_zlib_options are all derived from tune.*.* options. Plus, the tune_options object covers other tuning global directives and acts as a catch-all. Meanwhile, http_client_options and fifty_one_degrees_options are similarly derived from the manual's dot notation.

For the full and detailed specification of fields, please see the global model specification within our GitHub project.

Removed deprecated fields and resources

As a major version release, Data Plane API 3.0 gave us the chance to clean up keywords previously marked as deprecated. Here's more information about the keywords we've removed in v3:

Global section

• Removed the deprecated tune_ssl_default_dh_param field that can now be found in tune_ssl_options as default_dh_param

Backend section

• Removed the deprecated http-check field. You can now specify multiple http_check resources on /v3/services/haproxy/configuration/backends/{parent_name}/http_checks.

• Removed the http-keep-alive, http-server-close, and httpclose fields as they're mutually exclusive and configurable using the http_connection_mode field

HTTP request/response rules and TCP Request Rule resource

• Removed the deprecated track_sc<stick_counter> field. In v2 we only supported 0, 1, and 2 stick counter indexes by hard-coding them in the field names. We've now added the track_sc_stick_counter field (specifying the stick counter index) which can exceed 2 if configured using the tune.stick_counters keyword in the global section. It defaults to 3 and debuted in HAProxy 2.8.

Service discovery resources

• Removed the deprecated service-whitelist and service-blacklist fields, which we've replaced with service_allowlist and service_denylist, respectively

Further API Improvements

We'll describe other notable improvements in this section.

Fetching a parent resource with all children embedded

While we've found that our granular approach to creating resources on the API is helpful, we're aware it can also add complexity. As a result, we've added an extra query string parameter full_section on all the section resources such as frontend and backend resources. This lets you fetch, create, or replace a complete section with all of its child resources.

This query string parameter works on both PUT and POST endpoints, so you can now edit or create the whole parent section with just one API call.

Added a PUT operation to index based resources

Previously, change automation on a list of index-based resources was complex for several reasons. First, you had to reconcile all index fields in each resource. This made it difficult to locate a resource if something was deleted and added, as its index could have changed. Second, large lists with multiple changes forced you to make numerous API calls.

With v3, we added a PUT operation to the list endpoints, letting you automatically replace the entire list of resources.

Handle timeout options with preferred units

We handle timeout options as integers in the API, since we want to represent numeric values as numbers wherever possible. However, since HAProxy supports strings by accepting time unit suffixes, we recalculated the submitted values to the default unit in v2 (usually milliseconds) and stored them as integers in the configuration file. This was challenging for those who still use and read the raw configuration file, as it changed their preferred inputs and made the file less readable.

In v3, we've introduced a new option in the Data Plane API configuration file called preffered_time_suffix where you can specify one of the options: nearest, none, ms, s, m, h, and d. Data Plane API will then honor your preferred time suffix, which is nearest by default, and calculate it to the smallest possible value with the corresponding suffix. When using none, your timeout keywords in the configuration file are written without the suffix while using the default value for the respective keyword.

For further info, we've added some documentation to our Open API specification by annotating such fields with x-duration: true and x-default-unit to indicate the standard time unit for that specific field.

Keep Data Plane API config file unchanged

Since Data Plane API lacks its own storage, it previously used its configuration file to store state—notably for service discovery and user configurations. This proved problematic when using Data Plane API with automation software like Ansible, Puppet, and others.

Data Plane API will no longer write to its own configuration file from v3 onward. We've introduced dataplane_storage_dir, where Data Plane API's state will be stored in specific JSON files for each feature. We chose JSON for easier readability and debugging.

Upon startup, state data will automatically migrate from the configuration file into JSON files located at /etc/haproxy/dataplane. For information on where the data goes, view our brief Data Plane API internal storage Readme on GitHub.

Added support for all new keywords added in HAProxy 3.0

Following the latest release of HAProxy, we're extending the configuration keyword support to include many new features in the load balancer. These keywords impact security, persistence, streaming, directives, and more. 

To view and understand the complete collection of keywords now available in HAProxy 3.0, check out our Reviewing Every New Feature in HAProxy 3.0 blog post.

Contributions

As always, we'd like to extend a massive thank you to everyone who helped accelerate Data Plane API's development and make v3 possible: 

Contributor	Area
Zlatko Bratkovic	BUG | BUILD | CLEANUP | FEATURE
Olivier Duclos	BUG | FEATURE | REORG | TEST
Helene Durand	BUG | BUILD | FEATURE | REORG
Vincent Gramer	BUG | BUILD
Libo Huang	BUG
Andjelko Iharos	FEATURE
Marko Juraga	BUG | BUILD | CLEANUP | FEATURE | TEST
Dinko Korunic	FEATURE | OPTIM
Aurélien Maigret	BUG
Robert Maticevic	BUG | BUILD | CLEANUP | FEATURE | OPTIM | TEST
Fabiano Parente	BUG
Dario Tranchitella	BUG | REORG
Csaba Tűz	FEATURE
George Vine	BUG | FEATURE

Data Plane API 3.0 constitutes a major step forward in functionality and maintainability for Data Plane API. We're excited to bring you even more features and enhancements soon in response to your feedback. 

For more information including installation and upgrade instructions, visit our Data Plane API documentation. You can also view our Data Plane API releases section on GitHub for updated release notes and a changelog.

Subscribe to our blog. Get the latest release updates, tutorials, and deep-dives from HAProxy experts.