Dynamic updates

The Update module allows you to update the contents of ACL files, Map files, and TLS ticket key files periodically without reloading the load balancer. At startup, the process loads the contents of the files. Then, at a set interval, it downloads updates to the files from a specified URL. The content of the downloaded files replace the existing content.

This mechanism allows you to keep ACL and Map files up to date across multiple HAProxy Enterprise servers. Each server will check for changes independently.

Install the Update module Jump to heading #

1. Install the Update module according to your platform:

   apt yum zypper pkg

   nix
   sudo apt-get install hapee-2.8r1-lb-update

   nix
   sudo apt-get install hapee-2.8r1-lb-update

   nix
   sudo yum install hapee-2.8r1-lb-update

   nix
   sudo yum install hapee-2.8r1-lb-update

   nix
   sudo zypper install hapee-2.8r1-lb-update

   nix
   sudo zypper install hapee-2.8r1-lb-update

   nix
   sudo pkg install hapee-2.8r1-lb-update

   nix
   sudo pkg install hapee-2.8r1-lb-update

2. In the global section of your configuration file, add a module-load directive to load the Update module:

   haproxy
   global
     module-path /opt/hapee-2.8/modules
     module-load hapee-lb-update.so

   haproxy
   global
     module-path /opt/hapee-2.8/modules
     module-load hapee-lb-update.so

Update map files Jump to heading #

To update map files with the module:

1. Below the global section, add a new configuration section named dynamic-update. Inside, add one or more update lines that specify from where and how often to download new content. Here is an example that updates the contents of a file at /etc/hapee-2.8/redirects.map from the URL http://192.168.0.1:80/redirects.map every 5 minutes:

   haproxy
   dynamic-update
     update id /etc/hapee-2.8/redirects.map map url http://192.168.0.1:80/redirects.map delay 5m

   haproxy
   dynamic-update
     update id /etc/hapee-2.8/redirects.map map url http://192.168.0.1:80/redirects.map delay 5m

   As an example, the contents of this file below lists URL paths to redirect clients to during maintenance periods, based on the client’s IP address:

   redirects.map
   text
   10.0.0.0/8     /pages/maintenance-internal.html
   192.168.0.0/16 /pages/maintenance-external.html
   0.0.0.0/0      /pages/maintenance-catchall.html

   redirects.map
   text
   10.0.0.0/8     /pages/maintenance-internal.html
   192.168.0.0/16 /pages/maintenance-external.html
   0.0.0.0/0      /pages/maintenance-catchall.html

   Learn more about update

   The directive update supports the following syntax:

   text
   update id <id> url <url> [delay <delay> | xdelay <delay start next retry>] [timeout <timeout>] [retries <nb>] [modified | xmodified ] [source <addr>[:<port>]] [log] [dontlog-normal] [map] [purge <count> [<interval> [<id>]]] [tls-ticket-keys] [param*]

   text
   update id <id> url <url> [delay <delay> | xdelay <delay start next retry>] [timeout <timeout>] [retries <nb>] [modified | xmodified ] [source <addr>[:<port>]] [log] [dontlog-normal] [map] [purge <count> [<interval> [<id>]]] [tls-ticket-keys] [param*]

   where:

   Argument	Description
   id <url>	Required. The file name. The module uses the absolute file path. The file must exist when HAProxy Enterprise starts.
   url <url>	Required. Specifies the update URL.
   delay <u>	Specifies the period between each attempt to download a new file version. The delay is a simplified version of the xdelay keyword.
   xdelay <u s b r>	<u> specifies the period between each attempt to download a new file version; <s> specifies the initial (first) download delay; <b> is not used in this module and its valueis not important. However, its value must be present. Default: 10s; If the download fails, <r> determines the delay for the next attempt; Default values are: u = 5m, s = 5s, b = 10s, and r = 30s.
   timeout <t>	Specifies the HTTP connection timeout for attempts to download a new file version. The value is set in milliseconds by default, but you can set it to any other unit if you add a unit suffix to the number. Defaults to 5 seconds.
   retries <n>	Specifies the number of retries to download a new file. If not set, the global retries value applies (defaults to 3).
   modified	Specifies the use of the time from the Last-Modified response HTTP header. Example: checks whether to update the data using the If-Modified-Since request HTTP header.
   xmodified	Available since version 2.4r1. The same as modified except that the file modification time is set immediately after reading the file status.
   source <addr>[:<port>]	Sets the source address for outgoing connections. <addr> is the IPv4 address the load balancer binds to before it connects to a server; The default value is 0.0.0.0 to let the system select the most optimal address to reach its destination; <port> is optional; The default value of zero means that the system selects a free port; Does not support port ranges.
   log	Specifies whether to log operation errors.
   dontlog-normal	Deactivates logging of successful updates.
   map	Informs that the downloaded file must be interpreted as a map file. By default, the file is interpreted as an acl file.
   purge <count> [<interval> [<id>]]	This is available only for map and acl data types. Use the <count> argument to indicate that the module should delete old patterns when it downloads the next set of data. Otherwise, use <interval> and <id> to run a separate purge task. <count> is the number of old and/or invalid patterns that can be purged at once. Allowed values for that parameter are between 1 and 100000. The default is 1000; <interval> is the frequency for calling the pattern purging task. The value is set in milliseconds (if not specified by another time unit). If you specify the interval as 0, the task will be executed as quickly as possible. When setting the interval to 0, it is not recommended to purge a large group at once (a <count> of 100 is the optimal number for interval 0); <interval> has no default, so a value must be specified. <id> assigns a specific id to the purge task. You can use <id> to group certain purging tasks, or you can assign each operation its own task. The default is 1000, which means that all purging operations will be performed in the common task which has an id of 1000.
   tls-ticket-keys	Specifies that the downloaded file is a TLS ticket keys file (instead of an ACL or Map file).
   param*	Lists other server parameters that are useful for configuring SSL features.

   The options checksum and modified are mutually exclusive. If you define them at the same time, the option modified automatically switches off and a warning message prints when the load balancer starts.

2. In your frontend, use one of the map converters to read a value from the map file.

   For example, the map_ip converter will select the first matching row and cast it as an IP address. You can map a value to 0.0.0.0/0 as the last entry in the .map file to to act as a catch all for IP addresses that do not match any other patterns in the file.

   In the following snippet, a frontend references this data using an acl directive and an http-request redirect directive:

   haproxy
   frontend fe_main
     mode http
     bind :80
     acl redirect_needed src,map_ip(/etc/hapee-2.8/redirects.map) -m found
     acl redirect_performed path_beg /maint/
     http-request redirect location %[src,map_ip(/etc/hapee-2.8/redirects.map)] if redirect_needed !redirect_performed

   haproxy
   frontend fe_main
     mode http
     bind :80
     acl redirect_needed src,map_ip(/etc/hapee-2.8/redirects.map) -m found
     acl redirect_performed path_beg /maint/
     http-request redirect location %[src,map_ip(/etc/hapee-2.8/redirects.map)] if redirect_needed !redirect_performed

   If you specify a file for download via update in a dynamic-update block, you must make reference to this file somewhere else in your configuration (for example, in a frontend block, as in the snippet above shows).

   If you do not reference the file you specify in dynamic-update for use in another configuration section, the service will log an error similar to the following:

   nix
   [ALERT] (1525) : config : 'update': id '/redirects.map' not found in file ./haproxy.cfg at line 22
   [ALERT] (1525) : config : Fatal errors found in configuration.

   nix
   [ALERT] (1525) : config : 'update': id '/redirects.map' not found in file ./haproxy.cfg at line 22
   [ALERT] (1525) : config : Fatal errors found in configuration.

Update TLS session ticket keys Jump to heading #

When the load balancer and a client exchange messages over TLS, they share a secret key to encrypt their communication. This key is called a session key and it lasts only as long as the TLS session. Each new TLS session uses a unique key, which makes it difficult for an attacker to exploit.

Generating a session key costs the load balancer CPU time, which can add up when managing many clients. By asking each client to store their session key locally and reuse it between visits, the load balancer offloads some work and speeds up the time it takes to establish a TLS connection. The client can send their old session key to the load balancer to resume their session, negating the need to create a new one.

To make this secure, the load balancer uses a secret key known only to it to encrypt the session key in a package called a session ticket before sending it to the client. Only the load balancer can decrypt the ticket. The load balancer’s secret key should be changed at least every 24 hours in case an attacker steals it.

When you have multiple load balancers terminating TLS, you should ensure that TLS ticket keys are distributed across all of the load balancers. Otherwise, if a client makes a new connection with another HAProxy Enterprise server in the same cluster, a new key will need to be exchanged (with the associated CPU work).

In this section, we describe how to use the Update module to distribute the keys to all load balancers.

Configure your webserver Jump to heading #

Perform these steps on a web server that’s accessible to your HAProxy Enterprise servers:

1. Create a file named update_hapee_tls_tickets.sh and add the following contents to it. This script replaces the ticket keys stored in the file hapee_ticket_keys.txt. In this example, we use /var/www as the web server’s directory, but you can change this depending on your web server’s folder structure.

   nix
   #!/bin/bash
   openssl rand 80 -base64 >> /var/www/hapee_ticket_keys.txt
   new_keys=$(tail -n3 /var/www/hapee_ticket_keys.txt)
   echo "$new_keys" > /var/www/hapee_ticket_keys.txt

   nix
   #!/bin/bash
   openssl rand 80 -base64 >> /var/www/hapee_ticket_keys.txt
   new_keys=$(tail -n3 /var/www/hapee_ticket_keys.txt)
   echo "$new_keys" > /var/www/hapee_ticket_keys.txt

2. Run sudo crontab -e and add the following cron job to call the script every five minutes, changing the /path/to/file/ for your environment:

   nix
   */5 * * * * /bin/bash /path/to/file/update_hapee_tls_tickets.sh >>/dev/null 2>&1

   nix
   */5 * * * * /bin/bash /path/to/file/update_hapee_tls_tickets.sh >>/dev/null 2>&1

3. If possible, configure your web server so that only the HAProxy Enterprise servers can access the URL for hapee_ticket_keys.txt. If anyone else can get the contents of this file, they will be able to launch man-in-the-middle attacks against TLS connections to your load balancer servers.

Configure HAProxy Enterprise Jump to heading #

Perform these steps on your HAProxy Enterprise servers:

1. To give your load balancers a file with keys, which the Update module will later replace, create the file /etc/ssl/hapee_ticket_keys.txt. Add three ticket keys to it by running the following command three times:

   nix
   openssl rand -base64 48 >> /etc/ssl/hapee_ticket_keys.txt

   nix
   openssl rand -base64 48 >> /etc/ssl/hapee_ticket_keys.txt

2. Add a tls-ticket-keys argument to your HTTPS bind line:

   haproxy
   frontend ft
     bind :443 ssl crt /etc/ssl/site.pem  tls-ticket-keys /etc/ssl/hapee_ticket_keys.txt

   haproxy
   frontend ft
     bind :443 ssl crt /etc/ssl/site.pem  tls-ticket-keys /etc/ssl/hapee_ticket_keys.txt

3. Below the global section, add a new configuration section named dynamic-update. Inside, add an update line that specifies from where and how often to download new content. Here is an example that updates the contents of a file at /etc/ssl/hapee_ticket_keys.txt from the URL http://192.168.0.1/hapee_ticket_keys.txt every 2 minutes:

   haproxy
   dynamic-update
     update tls-ticket-keys id /etc/ssl/hapee_ticket_keys.txt url http://192.168.0.1/hapee_ticket_keys.txt delay 2m

   haproxy
   dynamic-update
     update tls-ticket-keys id /etc/ssl/hapee_ticket_keys.txt url http://192.168.0.1/hapee_ticket_keys.txt delay 2m

   After updating each load balancer server, each will query the web server at an interval and update the three keys in memory.

   As HAProxy Enterprise will use the middle of the three keys for encryption, slight variances in timing are acceptable, as the other HAProxy Enterprise servers will still be able to decrypt tickets from the other servers unless they miss two updates (since it’s running every two minutes, even in unfavorable conditions this should not happen).

   If you are transferring these keys over an untrusted network, the Update module supports HTTPS. Simply add the ssl and ca-file /etc/ssl/certs/ca-bundle.trust arguments to the update line, where ca-file points to your CA file bundle.

Runtime API Jump to heading #

It is possible to manage the Update module via the Runtime API.

lb-update list Jump to heading #

The lb-update list command returns the list of update lines in the dynamic-update section of the configuration file. Use lb-update list to display the files that the Update module manages.

nix
echo "lb-update list" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

nix
echo "lb-update list" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

output
text
# lb-update configured entries
# id: next update
/etc/hapee-2.8/redirects.map: 4m49s

output
text
# lb-update configured entries
# id: next update
/etc/hapee-2.8/redirects.map: 4m49s

lb-update force-update Jump to heading #

The lb-update force-update command launches an immediate update for the specified file.

nix
echo "lb-update force-update /etc/hapee-2.8/redirects.map" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

nix
echo "lb-update force-update /etc/hapee-2.8/redirects.map" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

output
text
lb-update: force-update `/etc/hapee-2.8/redirects.map`.

output
text
lb-update: force-update `/etc/hapee-2.8/redirects.map`.

lb-update status Jump to heading #

The lb-update status command shows the module status.

nix
echo "lb-update status" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

nix
echo "lb-update status" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

output
text
lb-update module status
------------------------------------------------------------------------------
 initialized:          yes
 
 SMP update
   configuration:      /etc/hapee-2.8/hapee-lb.cfg:30
   entry:              /etc/hapee-2.8/redirects.map
   generation id:      256
   url:                http://127.0.0.1:3000/redirects.map
   http status count:  0 44 0 0 0
   last modified:      Wed, 29 Mar 2023 19:13:45 GMT
   period/delay:       10s / 5s 10s 5s
   use modified:       no
   reload/retry count: 44 0 0 / 0 0
   reload time:        2023-03-29 19:13:49 / 2023-03-29 19:13:40 / <NEVER>
   download time:      2023-03-29 19:13:49 / <NEVER>
   next update in:     4s
   purging id(s):      #1000 0:20 1:23 2:23 4:23 6:22 8:20 11:20 16:17 253:16
      block size:       1
      interval:         1h

output
text
lb-update module status
------------------------------------------------------------------------------
 initialized:          yes
 
 SMP update
   configuration:      /etc/hapee-2.8/hapee-lb.cfg:30
   entry:              /etc/hapee-2.8/redirects.map
   generation id:      256
   url:                http://127.0.0.1:3000/redirects.map
   http status count:  0 44 0 0 0
   last modified:      Wed, 29 Mar 2023 19:13:45 GMT
   period/delay:       10s / 5s 10s 5s
   use modified:       no
   reload/retry count: 44 0 0 / 0 0
   reload time:        2023-03-29 19:13:49 / 2023-03-29 19:13:40 / <NEVER>
   download time:      2023-03-29 19:13:49 / <NEVER>
   next update in:     4s
   purging id(s):      #1000 0:20 1:23 2:23 4:23 6:22 8:20 11:20 16:17 253:16
      block size:       1
      interval:         1h

The fields of the lb-update status command are described below:

lb-update purge Jump to heading #

Deletes data from the list of old/invalid patterns for the specified file. Use lb-update purge to run a purge task immediately.

nix
echo "lb-update purge /etc/hapee-2.8/redirects.map 13 1" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

nix
echo "lb-update purge /etc/hapee-2.8/redirects.map 13 1" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

output
text
lb-update: purge  `/etc/hapee-2.8/redirects.map` 13 : all pattern(s) purged!

output
text
lb-update: purge  `/etc/hapee-2.8/redirects.map` 13 : all pattern(s) purged!

Two additional parameters are optional for purge:

• <gen_id>: the pattern generation id
• <count>: the number of records to purge

If no <gen_id> is specified, unnecessary patterns are deleted in the order in which they were added to the list. The default value for <count> is 1000 and the allowable range is between 1 and 100000.

Use the lb-update status command to find the pattern generation id (<gen_id>). These ids may be listed under purging id(s) where generation ids are listed with the number of patterns ready to purge per id.

nix
purging id(s): #1000 0:20 1:23 2:23 4:23 6:22 8:20 11:20 16:17 253:17

nix
purging id(s): #1000 0:20 1:23 2:23 4:23 6:22 8:20 11:20 16:17 253:17

If, for a particular generation id, there remain additional patterns to purge, the number of remaining patterns is shown:

nix
echo "lb-update purge /etc/hapee-2.8/redirects.map 13 100" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

nix
echo "lb-update purge /etc/hapee-2.8/redirects.map 13 100" | sudo socat stdio unix-connect:/var/run/hapee-2.8/hapee-lb.sock

output
text
lb-update: purge  `/etc/hapee-2.8/redirects.map` 13 : 100 pattern(s) purged, 200 left!

output
text
lb-update: purge  `/etc/hapee-2.8/redirects.map` 13 : 100 pattern(s) purged, 200 left!