5 Ways to Extend HAProxy With Lua

Use Lua to add fetches, converters, actions, services and tasks to HAProxy.

Did you know that HAProxy embeds the Lua scripting language, which you can use to add new functionality?

HAProxy features an extremely powerful and flexible configuration language and gives you the building blocks you need to handle many complex use cases. However, at certain times, you may want to extend HAProxy to meet a unique scenario that isn’t addressed. While HAProxy itself is written in the C programming language, and you can extend it with C—contributions are appreciated—for many of us, starting with a scripting language is a much easier proposition.

Lua is a lightweight programming language that was created by a team of researchers at the Pontifical Catholic University in Rio de Janeiro, Brazil. The meaning of its name is Moon in Portuguese. It’s implemented in C, making it portable and embeddable and you’ll often find it used for scripted add-ons and applications, especially for game development. It’s perfect for extending HAProxy! The syntax is also short and concise, which makes it a joy to use. The official Lua site links to many resources that will help you get up to speed on its use.

The purpose of this blog post is to show you how to use Lua to extend HAProxy. We’ll explain what’s possible to do, and what’s not, or at least maybe not recommended. Sometimes you might even prototype new HAProxy functionality with Lua and then decide whether you want to reimplement it in C for speed—although Lua is pretty fast for a scripting language!

We’ll demonstrate some code examples, but see our Lua code repository for more context. Before trying the examples, make sure that your HAProxy binary was compiled with Lua support. Lua first appeared in HAProxy 1.6, but we recommend that you upgrade to the latest version if possible. Use the following command to check whether Lua support is included:

Lua Says: Hello HAProxy

To respect the tradition, we’ll start with the simplest possible script. This will give us an opportunity to show you how to load your scripts so that HAProxy can use them. Open your favorite text editor and create a file named hello.lua. Add the following line:

Then, edit your HAProxy configuration, adding a lua-load directive in the global section:

As you no doubt guessed, lua-load loads a Lua file when HAProxy starts or is reloaded. Try it out by invoking HAProxy from the command line, like this:

You should see Hello HAProxy! printed to the screen. You can have multiple lua-load directives. They will be executed sequentially. You can also use the require function to load other Lua files from within your script, like you would for any standard Lua project.

The example above uses the Debug function from the core class. When writing Lua scripts that read input or write output, it’s usually a good idea to use the classes provided by HAProxy, which are compatible with HAProxy’s non-blocking architecture, rather than the standard Lua input/output functions. So, we’ve used the core Lua class to write something to the HAProxy log.

It is safe, however, to read and write to files using the standard Lua functions during the script’s initialization phase. For example, you could add a register_init function that does this, as we’ve done in the JWT library for HAProxy.

In addition to lua-load, there are a few tuning directives that can be added in the global section. Search for the keyword tune.lua in the HAProxy configuration manual.

HAProxy Extension Points

When you use Lua with HAProxy, your scripts get access to a library of classes specific to the task of extending HAProxy. This allows you to add any of the following:

• Fetches

• Converters

• Actions

• Services

• Tasks

You can use these to hook into different parts of the processing pipeline. We’ll cover all of the above, describing related helper classes.

Fetches

A fetch is a piece of information about a connection, request, or some internal state provided by HAProxy. For example, the src fetch method returns a client’s source IP address. Fetches are valuable when making decisions, such as where to route a request, whether to deny a client, or whether to process the message in a special way such as by compressing it. We’ve covered using fetches in our blog post Introduction to HAProxy ACLs.

With Lua, you can create your own. Generally speaking, you can’t return totally new information with a Lua-defined fetch, but you can combine existing methods and fetches to create something new and useful. The syntax for creating a fetch method in Lua is the following:

You’d use the fetch in your HAProxy configuration as lua.foo_fetch. The foo function is passed a txn object, followed by optional input parameters that you can pass to the fetch. You can use txn to call other fetches or converters, use channels, and so on, from inside your function. The txn class exposes almost all standard HAProxy fetches as:

The txn class exposes many other useful objects and methods, either for the current request/response or for more generic HAProxy functionality. The following table lists some that are particularly interesting:

Function or object	What it does
txn.req and txn.res	Get the request and response channels.
txn.f and txn.c	Returns collections of existing fetches and converters.
txn.http	A class that’s exposed in proxies using HTTP mode. This helper class can be used to manipulate request/response attributes, such as the request line, method, path, URI, etc. This saves you from having to implement HTTP parsing yourself.
txn.get_var(name) and txn.set_var(name, value)	Get or set a variable in HAProxy.
txn.set_priv(data) and txn.get_priv()	Gets or sets data that you want to pass to other parts of your Lua code during a single transaction. It’s similar to setting a variable with transaction scope.

Use the register_fetches function to add your new fetch to HAProxy. Its first parameter defines what the fetch will be called in the configuration and the second maps to the function that will be invoked.

To give an example of a custom fetch method, let’s create one that compares two variables. Create a file called greater_than.lua and add the following code:

This code gets the value of two HAProxy variables, whose names are passed in as var1 and var2, by using the txn.get_var function. It then checks whether the first holds a number that’s greater than the second’s and, if so, returns true. Otherwise, it returns false. Also, notice how you can define the function inline with the core.register_fetches function. You might use this to compare a client’s connection rate with a threshold when both are stored as variables.

Note that you can also compare two variables in a less obvious way by using an ACL statement like this, without Lua:

Next is a more advanced example that makes uses both the txn.f class to fetch a frontend name and the server class’ get_stats function to get statistics. This custom fetch tells you which backend has the fewest active sessions. Create a file called least_sessions.lua and add the following code:

This code will loop through all of the backends that start with the same letters as the current frontend, for example finding the backends www_dc1 and www_dc2 for the frontend www. It will then find the backend that currently has the fewest sessions and return its name.

Use a lua-load directive to load the file into HAProxy. Then, add a use_backend line to your frontend to route traffic to the backend that has the fewest, active sessions.

Converters

Sometimes, when a fetch returns a piece of information, you need it transformed before you can use it in a practical way. If the existing HAProxy converters are not enough, you can use Lua for that transformation. The syntax for creating a converter is the following:

Use the register_converters function to inform HAProxy that your converter exists. Its first argument is the name you’d like to use in your HAProxy configuration and the second is the function to invoke.

For the next example, create a new file called urlencode.lua. Within it, add the following code, which defines a converter that URL encodes a passed-in value:

You might use this to encode a URL before redirecting a client to it. The converter can be referenced in your configuration as lua.urlencode. Update your configuration so that it contains the following:

For this example, we use a hardcoded string, Vinyl & Rare Music, as the company name, but you could also get it from a map file. After encoding the name, it’s set as the URL path and the user is redirected to http://192.168.50.20/Vinyl+%26+Rare+Music.

Actions

Actions give you a way to modify L4 and L7 messages. With actions you can accept or reject TCP connections, add HTTP headers with dynamic values (such as the Access-Control-Allow-Origin and Access-Control-Allow-Methods headers needed for CORS), and rewrite the request or response’s URL path, query parameters, or HTTP status. You’d use the following directives to invoke your custom action in your HAProxy configuration:

• tcp-request connection <action>

• tcp-request content <action>

• tcp-response content <action>

• http-request <action>

• http-response <action>

We touched upon Lua actions in our previous blog post Using HAProxy as an API Gateway, Part 2, so make sure you check it out. Another nice example is haproxy-auth-request from Tim Düsterhus. The Lua syntax for defining an action is:

Use register_action to add your new action to HAProxy. Its first parameter is the name of the action. The second is a list of applicable directives where this action can be used. Typically, you would limit your action to apply to either TCP or HTTP and also to either a request or a response. The third parameter identifies the function to invoke, which, in this case, is foo. The final parameter is the number of additional arguments that foo accepts. In this case, it’s zero. You would invoke the action like this:

If you need to return a value to HAProxy after the action is invoked, you can set a variable. Then, you can reference that variable in ACL statements.

In contrast to converters and fetches, your Lua actions can and will often use socket functions, which allow them to communicate with external services. The Socket class is a replacement for the standard Lua Socket class and is compatible with HAProxy’s non-blocking nature. When you want to use socket functions in your actions, you must use this class.

An instance of the Socket class is retrieved by calling core.tcp(). Then you can use the following methods:

Function	What it does
Socket.connect(address, [port])	Connects to the specified address and port.
Socket.connect_ssl(address, port)	Connects to the specified address and port using TLS.
Socket.close()	Closes the open socket.
Socket.settimeout(value, [mode])	Sets the socket timeout. This should be lower than the Lua session and service timeouts.
Socket.send(data, [start], [end])	Sends data over the socket connection.
Socket.receive([pattern], [prefix])	Receives data over the socket connection.

To give you an example, let’s say that you wanted to extend HAProxy by checking the client’s source IP against a registry of banned IPs. Our action will make a remote call to an IP Checker service and then set a variable, req.blocked, to true if the client should be denied access. Note that you can also define lists of whitelisted or blacklisted IP addresses by using HAProxy ACL files, but this example allows us to demonstrate the socket functions.

The IP Checker service will be a Python script that returns, randomly, either allow or deny. Remember, these examples are available in our code repository. First, install Python and Flask:

Add a file named ipchecker.py with the following code:

Then run the application by using the flask run command:

In another window, try making a request to the IP Checker service using curl. Append an IP address as the path. It should return either allow or deny.

Next, let’s add the Lua code that will call this service. Create a file called ipchecker.lua and add the following code:

Notice that we’re using the function core.tcp() to get an instance of the Socket class. We can use it to connect to the Python service and send requests. Once we have the response, we set a variable named req.blocked to true or false, depending on whether the content of the response was allow. You are also able to access fetch methods, such as txn.f:src() to get the client’s source IP address. Next, update your HAProxy configuration so that it loads the Lua file and uses our new action.

Requests will now be randomly denied. Notice that we’re passing two parameters to lua.checkip: the IP address and port of the Python Flask service. When we added the action using core.register_action, we set the last parameter, which declares the numbers of parameters that the action expects, to two.

Services

With services, also known as applets, you can ask HAProxy to deliver a requests to your Lua script, which will then generate the response. No backend servers will be contacted. This is handy if you want to add some application logic to your proxy. Or, maybe you long for the old days of Apache + mod_php scripts. Who knows!

Like actions, services are free to use socket functions (subject to HAProxy timeouts). The syntax for creating services with Lua is the following:

A service can be invoked from your HAProxy configuration by passing its name to http-request use-service, as shown:

The mode passed to core.register_service can be either http or tcp. Depending on that, the function, in this case foo, will receive an AppletHTTP or AppletTCP object, respectively. The applet objects provide you with enough functionality to create mini services with Lua.

The AppletHTTP class has attributes and methods useful for working with an HTTP request and response. The following table outlines them:

Function or object	What it does
AppletHTTP.method	Gets the request method.
AppletHTTP.path	Gets the request path.
AppletHTTP.qs	Gets the query string.
AppletHTTP.length	Gets the request body length.
AppletHTTP.headers	Gets a table of request headers (zero-based table).
AppletHTTP.receive([size])	Reads the data from the request body.
AppletHTTP.add_header(name, value)	Adds a header to the response.
AppletHTTP.set_status(code, [reason])	Sets the response status.
AppletHTTP.start_response()	Starts the response (send the headers).
AppletHTTP.send(message)	Sends the response body.

The AppletTCP class is simpler than AppletHTTP. The following table outlines its available functions:

Function	What it does
AppletTCP.receive([size])	Reads data from a TCP stream.
AppletTCP.getline()	Reads a single line from a TCP stream.
AppletTCP.send(message)	Sends data on a TCP stream.

To demonstrate a custom service, we’ll create one that returns Magic 8-ball answers directly from HAProxy. Create a file named magic8ball.lua and add the following code to it:

In your HAProxy configuration, use http-request use-service to invoke the service if the URL path is /magic. Otherwise, route traffic normally.

Now, when a client makes a request to /magic, they’ll get a random Magic 8-ball response. It’s interesting to see that, unlike a request to a backend, which records the backend name in the access log, when the service is invoked the log shows <lua.magic8ball> as the backend name.

Tasks

Last but certainly not least, you can add functions that run in the background, in the spirit of cron jobs. The syntax is very simple:

You might utilize tasks to write complex health checks, for example. However, as a simple example, this will write Doing some task work! to the HAProxy log every 10 seconds:

Save this as log_work.lua and then use lua-load to add it to HAProxy. You don’t need to add anything other than a lua-load directive to register and start a task. From your task, you can use any variables that were declared in the script’s global scope. You can also reference HAProxy variables and use socket functions to connect to external services.

Conclusion

In this blog post, we demonstrated how to expand HAProxy’s functionality by writing custom Lua code. Methods exist for creating your own fetches, converters, actions, services, and tasks. This means that you can extract new combinations of data from requests, transform that data, communicate with external services, respond directly from HAProxy, and create long-running, background tasks. Opportunities are plentiful for finding new and interesting ways to extend your load balancer.

Want to keep up to date on similar topics? Subscribe to this blog! You can also follow us on Twitter and join the conversation on Slack. HAProxy Enterprise includes a robust and cutting-edge codebase, an enterprise suite of add-ons, expert support, and professional services. It also includes modules that have been developed to extend HAProxy in complex and powerful ways. Want to learn more? Contact us today and sign up for a free trial.

Subscribe to our blog. Get the latest release updates, tutorials, and deep-dives from HAProxy experts.