Discovering the Versatility of HAProxy Enterprise with Willhaben
More than 10 million listings.
Team of 250 staff.
2.5 billion page impressions per month.
About Willhaben
Willhaben is Austria’s largest online marketplace, offering ads for goods and services ranging from careers to real estate. First brought online in 2006, the team of 250 has grown the site to now advertise almost 10 million items at any one time, making it one of the most visited in the country. And with its 2.5 billion page impressions per month, Willhaben has strict requirements on the safety and efficiency of its network.
Results at a Glance
The Challenge
Over the first decade of its life on the web, Willhaben’s online demands matched the steady growth of its small startup team. Meaning that with just 600-800 megabits per second of traffic in 2015 delivered mostly through HTTP, it was able to get by with a limited, hardware-based load balancing solution to route packets to its four pools of servers. But with greater traffic on the horizon and the incentive of reinforced overall security, Willhaben took advantage of the lesser costs of a Linux-based approach and became an HAProxy Enterprise customer.
Systems engineer Christian Platzer’s initial experience with the solution proved immediately that it was more secure, as he was now able to implement HTTPS in the load balancer, which also offered superior performance given that HAProxy Enterprise allows the assignment of SSL encryption work to specific CPU cores. This ability to pin SSL encryption to specific cores gave Willhaben an efficient means to implement a greater level of security throughout the site.
But Willhaben soon discovered that this enhanced security was only one of the many tools on the swiss army knife that is the HAProxy Enterprise solution to load balancing.
Christian Platzer at HAProxyConf 2019
The Objectives
With growing traffic and a desire for the on-demand scalability and savings of a move to Kubernetes, Willhaben used a self-written Ingress Controller utilizing HAProxy Enterprise. They then took advantage of the hitless configuration file reload capability in HAProxy Enterprise 1.9 to support the dynamic addition and removal of Kubernetes services.
The team of engineers also wanted to implement HTTP/2 across the site, chasing the efficiency enabled by the protocol’s ability to send multiple client requests in parallel on the same TCP connection. They discovered that this switch was relatively seamless with easy configuration in HAProxy Enterprise and were also able to take advantage of the Enterprise support team when they discovered a series of busy looping processes associated with the update, as well as the known Race Cache with Network issue in Firefox. The Enterprise support team was quickly able to help Willhaben with a fix, and the company was ready to face their final security challenges.
The Solution
Christian’s final goals for HAProxy Enterprise were the utilization of its security features to enable a solution to one of his personal vendettas: Search Crawler Protection. While Willhaben wants its classifieds to be crawled by Google, the team has encountered issues with Google and Bing bot requests, as well as other crawlers interfering with access number data for advertisement impressions. While from the start Christian was implementing HAProxy Enterprise’s tarpitting function to return 502 Bad Gateway or 503 Service Unavailable errors to block specific IP addresses, he found even greater effectiveness with its Javascript Challenge Module.
This tool on HAProxy Enterprise provides the ability to ask the client’s browser to solve a math-based challenge to prove that it is a real browser. This avoids some issues with accidentally tarpitting valid clients running behind a NATed network or web proxy, or mobile clients coming from a dynamic IP pool.
The Results
Six years after switching to HAProxy’s Enterprise solution to load balancing, the Willhaben team continues to see improvements to their system with each release. They are also taking a keen interest in HAProxy Technologies’ dialogue with the community over upcoming updates to the software, including traffic shadowing capabilities (which has since been released), all while enjoying the proven security benefits of software-based load balancing.
What HAProxy Enterprise Offers You
While Willhaben implemented a custom solution to Ingress Control with HAProxy Enterprise, you can now get many of the same benefits without doing so yourself. HAProxy Technologies now also offers an enterprise ingress controller, meaning support for all the benefits associated with containerized management, continuous delivery, and more. Contact us to learn further.
Interested to learn more about HAProxy use cases? Explore our Success Stories page.