Building a Service Mesh at Microsoft Yammer with the power of HAProxy Enterprise
More than 2 million requests per second at any point in time.
One of the most used enterprise social networking solutions with over 85% Fortune 500 companies using it.
Founded over 12 years ago.
About Microsoft Yammer
Yammer is a social networking tool, part of the Microsoft Office 365 suite, used to openly connect and engage across an organization. With Yammer, you can start conversations that everyone in your organization can join, announce news, find groups by topic, and share documents. Its aim is to connect people and information to enable better decisions, faster.
The Challenge
When Microsoft acquired Yammer in June 2012, the team migrated its services to Microsoft data centers. While traditional data centers were adequate for a time, the team, led by Tobias Haag, Software Engineer Lead, decided that moving to Microsoft Azure would better serve their needs through the use of instant, on-demand scale, massive geographic reach, and the chance to offload the complexities of networking. However, they wanted to build additional failsafes to guard against downtime and, for that, they landed on the idea of designing a platform on top of Azure that would increase their resilience, accelerate developer velocity, and comply with security and governance regulations.
Having moved away from traditional data centers and looking to displace legacy hardware load balancers, Yammer looked to HAProxy as a Layer 7 solution. Tobi and the team realized the extensive value of HAProxy and wanted to explore using it across their Azure environment for both internal and external applications. It had demonstrated clear advantages over other load balancers that they’d used in the past. Now, they wanted to expand upon it to see if they could utilize HAProxy even further within their new architecture layered over Azure.
Tobias Haag onstage during his HAProxyConf 2019 talk
The Objectives
Tobi’s priority for the Yammer platform was robust reliability. While Azure managed networking, DNS & server provisioning as a service, he wanted to add auxiliary mechanisms to mitigate, and ultimately altogether prevent customer-impacting outages.
They also wanted to build self-service infrastructure, wherein developers could spin up new applications quickly, at any scale. Since the environment would be so dynamic, they needed service discovery, efficient health checks, and a way to wire up interdependent applications automatically. All of this would lead to improved developer productivity.
The last objective was to incorporate security and compliance measures. Most importantly, they wanted to encrypt communication between all of their applications, not just the connections coming in from the client. Enforcing SSL across their organization would mean consistent security everywhere.
The Solution
Yammer’s new platform leveraged Docker containers to provide highly resilient uptime across a cluster of servers. It used Mesos and Marathon to orchestrate these containers, and HAProxy Enterprise to relay all communication between them.
The Yammer team already preferred HAProxy for routing requests at the edge of their network. However, with a large number of applications in the mix, routing east-west traffic through that same set of load balancers seemed inefficient. Their new platform allowed them to create a large scale service mesh with an instance of HAProxy Enterprise next to each containerized application, enabling direct communication between services and providing health checks, SSL encryption, DNS-based service discovery, and observability at the container level.
HAProxy would continue to serve as the Layer 7, edge load balancer in front of their platform, enabling secure and reliable communications to external clients. However, it would now assume a new role and provide a range of load balancing features to all of their internal services.
Yammer’s infrastructure utilizes dedicated HAProxy Enterprise clusters, running around 60,000 HAProxy instances
An important consideration was the time needed to spin up HAProxy. “It turns out that HAProxy, just spinning it up is extremely cheap for us,” said Tobias Haag. “The amount of overhead that we get compared to our applications is almost negligible. In steady state, HAProxy doesn’t use any CPU, or any noticeable CPU.”
The Results
HAProxy Technologies proved to be a great partner for Yammer as they migrated to a Microsoft Azure-based microservices architecture with the help of our support team, leveraging our Technical Account Manager services to their fullest. Today, Yammer runs a custom service mesh of 60,000 HAProxy Enterprise instances across their stack with 2,000,000 requests per second flowing through the system at any point in time.
“Moving to a software-defined load balancer, and HAProxy Enterprise in particular, has allowed us to implement an orchestration-bound system that can scale dynamically and deliver the performance and reliability that our users demand in a very cost-effective manner,” said Haag. “HAProxy Enterprise allows us to very reliably load balance between all of our components. It allows us to handle all SSL terminations while updating configurations very easily. It provides a ton of flexibility that saves us significant development time. Now, we can focus our time on making the overall developer experience better and building out new features for our customers.”
Interested to learn more about HAProxy use cases? Explore our Success Stories page.