A recent vulnerability in the HTTP/2 protocol could allow denial-of-service (DoS) attacks by exploiting the protocol's CONTINUATION frame to flood web servers, reverse proxies, or other software processing HTTP/2 traffic.
After rigorous testing, we have confirmed that our implementation of the HTTP/2 protocol can effectively handle the CONTINUATION Flood. Considering HAProxy was built from the ground up to withstand DoS attacks, its resilience to the HTTP/2 CONTINUATION Flood is no surprise. We will continue to monitor, but the supported versions of our products are not vulnerable to the known attack vectors.
What’s an HTTP/2 CONTINUATION Flood?
To understand how an HTTP/2 CONTINUATION Flood functions, we first need to understand how the HTTP/2 protocol facilitates multiplexing.
The HTTP/2 protocol is designed for multiplexing, a capability facilitated by the breakdown of communication into smaller units known as a “frame”. HTTP/2 frames are the smallest unit of communication in the protocol, serving a specific purpose, such as carrying headers or data. Each frame embeds a unique stream ID that identifies a stream of communication between two peers. Multiple streams can be opened in parallel between a client and a server—and this is called “multiplexing”. All streams and frames are exchanged over the same TCP connection.
HTTP/2 supports many frame types, but for the purpose of this discussion, we’ll focus on two: HEADERS and CONTINUATION.
In order to better understand how this works, let’s explore how an HTTP/2 connection is established:
The client and server establish a TCP connection.
The client and server negotiate an SSL connection.
The client and server exchange a SETTINGS frame to negotiate various parameters of the HTTP/2 connection, including “SETTINGS_MAX_FRAME_SIZE”.
The client sends a HEADERS frame with HTTP header fields.
If the HTTP/2 header size is bigger than the negotiated “SETTINGS_MAX_FRAME_SIZE”, a CONTINUATION frame is sent with additional header fields.
Repeat step five until the whole HTTP/2 header is transmitted.
The last CONTINUATION frame has a special flag set: END_HEADERS, which means the client sent the data it was supposed to.
All HEADER and CONTINUATION frames are sent using the same stream ID, as mentioned above.
There’s an element of fun here: the client can decide the amount of data it sends in the HEADERS and CONTINUATION frames. In an extreme example, a client may send 8192 frames (1 HEADERS and 8191 CONTINUATION), each containing 1 byte of data, to send an 8KB request HTTP header. This situation initiates a flood. Despite being impolite, this flood is legitimate.
As outlined in the steps above, a server should wait until it gets the END_HEADERS flag. However, in the context of an attack, the perpetrator withholds this flag and continuously sends CONTINUATION frames with dummy data. The attacker’s goal is to exhaust the server’s memory and kernel resources to kill the process with an Out Of Memory error. Voilà!
Why is HAProxy resilient to the HTTP/2 CONTINUATION Flood?
Fortunately, HAProxy’s implementation of the HTTP/2 protocol is resilient to the HTTP/2 CONTINUATION Flood. Let’s reexamine the steps above to understand how this is handled by HAProxy:
The client and server establish a TCP connection.
The client and server negotiate an SSL connection.
HTTP/2 SETTINGS is negotiated, and the SETTINGS_MAX_FRAME_SIZE is set to HAProxy’s “tune.bufzise” (16KB by default, which is also the minimal value for this H2 setting).
The attacker sends the HEADERS frame. HAProxy allocates a buffer dedicated to this new stream ID and copies the data in there.
The attacker floods CONTINUATION frames, and HAProxy writes the data in the dedicated buffer for this stream ID.
The dedicated buffer is full (but the attacker is continuously flooding HAProxy).
Because the buffer is full and does not contain (yet) a full and valid HTTP request, HAProxy returns an HTTP 400 status code error message and closes the stream ID.
During that time, the server is not aware that an attacker is trying to abuse the service. That’s the beauty of a reverse proxy (but we’ll go into deeper details in the next section).
An attacker may assume that a CONTINUATION Flood could overwhelm HAProxy, especially if the CONTINUATION frames send just 1 byte of data. However, HAProxy is known for its high performance, capable of handling 1,000,000 CONTINUATION frames per second per CPU core (depending on your type of CPU). Ultimately, the attacker is unable to overwhelm HAProxy, making HAProxy the reliable solution users can trust when they expose their applications on the internet.
We can take it a step further and completely disconnect a TCP connection when we consider a client is abusing the H2 protocol. Set the "tune.h2.he.glitches-threshold" parameter to detect this (and other kinds of H2 attacks like the Rapid Reset Attack). HAProxy’s flexibility and observability mindset allow you to conveniently log the glitches counter associated with clients by using the "fc_glitches" fetch output to help you set the parameter above.
How can HAProxy, as a reverse proxy, protect my application?
HAProxy's innate ability to operate as a reverse proxy offers a formidable defense against attacks like HTTP/2 CONTINUATION Flood. In HTTP terms, a reverse proxy is a gateway standing between the client and server, breaking the communication into two isolated connections.
1. One connection with the client
2. One connection with the server
When a connection is formed, the client communicates directly with HAProxy rather than the server. The client can only see HAProxy and its robust defense. The server is shielded behind HAProxy, and its vulnerabilities remain unexposed to the Internet.
This means HAProxy is well-positioned to defend against threats before they reach the server. HAProxy's security capabilities bolster its resilience against attackers, ensuring that servers are safe.
As a reverse proxy, HAProxy can handle an HTTP/2 CONTINUATION Flood without the server being aware that an attack is taking place. Therefore, if you're using HAProxy in front of applications vulnerable to HTTP/2 CONTINUATION Flood attacks, there's less pressure to update components.
If your components are vulnerable to the flood, you have two options: fix them very quickly or install HAProxy in front of them.
Related Articles:
Conclusion
HAProxy engineers worked alongside the community in shaping the design of the HTTP/2 protocol, ensuring that when it came time for implementation, HAProxy would be future-proofed against potential threats like the HTTP/2 CONTINUATION Flood.
Users love HAProxy because it’s reliable, performant, and flexible—and this is demonstrated in its resilience to DoS attacks. With HAProxy, there is no compromise—you get all three.
HAProxy is trusted by leading companies and cloud providers publishing services and APIs on the Internet. Its resilient, high-performance architecture and robust, peer-reviewed open source codebase make it one of the most trustable layers in your application delivery stack.
Subscribe to our blog. Get the latest release updates, tutorials, and deep-dives from HAProxy experts.
