Load Balancer Explained - System Design 07

What is a Load Balancer?

Usually in a large data center, there may be millions or even tens of millions of requests from other servers or clients per second. In order to effectively handle this traffic, we must manage and balance this traffic, and the load balancer is such a module, specially designed to handle this kind of large traffic problem!

The load balancer mainly distributes all user requests to the corresponding servers based on the current server load. The advantage of this is to avoid server overload or crash. However, you can also think about it here. If the traffic of the current system is not high, about only a few thousand requests per second, then a load balancer may not be needed. After all, the more complex a system is, the more things need to be considered.

Load balancer features

If a load balancer is an element designed to handle large traffic, what features should the load balancer need to have?

Scalability (Scalability)

according to Introduction to Modern System Design - System Design 01 It was mentioned that a good system must consider scalability, because the larger the system, the more servers or components it requires. The load balancer has the characteristic that it is a component that can be expanded or reduced, so it can be expanded according to the current system.

Availability (Availability)

The so-called availability means that if some servers of the system crash, the entire system will not stop functioning. Load balancers can effectively handle this type of problem, because load balancers can determine which servers are overloaded or down, and redirect traffic to servers that are operating normally, thereby avoiding system failures or even outages.

Performance

For a load balancer, a very important task is to ensure that each server is evenly distributed. Basically, there will be no problem of excessive traffic on a specific server. Such a feature can ensure that the user can get a faster response time.

How does a load balancer work?

Let’s first take a look at how the server operates without a load balancer.

As mentioned before, without a load balancer, traffic will be directed to a specific server and the client will be delayed.

On the contrary, if a load balancer is placed in the middle, the traffic can be effectively balanced to the corresponding server.

AWS Load Balancer Explained

Here we provide the schematic diagram of AWS to explain, and attach the source and source!

The following is the translated text of AWS’s original text:

Companies often run their applications on multiple servers. This type of server arrangement is called a server farm. User requests to the application first go to the load balancer. The load balancer then routes each request to the single server in the server cluster that is best suited to handle the request.

Load balancing is like what a restaurant manager does. Consider a restaurant with five waiters. If customers are allowed to choose their waiters, one or two waiters may be overloaded while others sit idle. To avoid this, restaurant managers assign customers to specific waiters who are best suited to serve them.

How a load balancer works

Load balancing is handled by the load balancer, so it can be divided into two parts. The first is the load balancer tool, and the second is the matter of load balancing.

Load balancer

The first is that load balancers can be hardware or software. If it is a hardware load balancer, you need to install a dedicated load balancing device, while if it is a software load balancer, it can operate on servers, virtual machines, and clouds. once in Introduction to System Design Components Building Block – System Design 05 The content delivery network (CDN) mentioned usually also has a load balancing function. In the future, I will spend an article explaining CDN properly.

Load balancing algorithms

When a user sends a request, the load balancer passes the request through different algorithms to determine which server to handle the request. These algorithms can be divided into two major categories: static load balancing algorithms and dynamic load balancing algorithms.

Static load balancing algorithms

The static load balancing algorithm performs load balancing based on the average algorithm and does not consider the current system status. To put it bluntly, static load balancing does not know the status of the server, and it cannot decide whether to import traffic based on whether the server is overloaded or idle. Instead, the load is distributed through the plan based on it from the beginning. The advantage of this is that it is very easy to set up and understand load balancing, but of course the disadvantage is that it will lead to low efficiency.

One of the common static load balancing methods is Round Robin (RR). This algorithm is actually quite common in operating systems and CPU scheduling. As the name suggests, it executes corresponding tasks in turn based on the amount of time. In order to explain the Round Robin algorithm in more depth, here is also providedWikipediaexample.

In a system with a time slice length of 100 milliseconds, consider the processes listed in the following table:

Dynamic load balancing algorithms

The dynamic load balancing algorithm will consider the current status of each server, including: availability, workload conditions, and server operation conditions.

And it will transfer traffic from overloaded or underperforming servers to servers that are not fully loaded. This has the advantage of evenly distributing traffic more efficiently. However, the disadvantage is that dynamic load balancing is difficult to achieve. Many different factors will affect the availability of the server, such as: server operation, server memory capacity, and the size of client requests.

Here we also provide several common dynamic balancing algorithms for readers, including Least Connection, Weighted Least Connection, Resource-Based Load Balancing, and Geographical Location Load Balancing. (Geolocation-based Load Balancing)

Global and local load balancing

We mentioned earlier what a load balancer is and how it works. Next, let’s talk about the types of load balancers. They can basically be divided into global load balancers (Global Server Load Balancing) and local load balancers (Local Load Balancing). ). If you simply take the literal meaning, it is not difficult to see that global load balancing is responsible for traffic balancing across multiple geographical locations, while local load balancing is responsible for traffic balancing in a large data center.

Global Server Load Balancing

The main task of global server load balancing is to balance and distribute global traffic to different large data centers. For example: US user traffic is distributed to data centers in North America through GSLB. Global server load balancing can decide which geographical data center to direct traffic to based on the user's current geographical location and the status of different data centers.

Here's another example: if a user is in Tokyo, and they're using a web service with servers in New York, both the request and the response have to travel a long distance, which causes significant delays in load times.

Using GSLB, global server pooling ensures that each user can connect to a server that is geographically close to them, minimizing hops and transfer times. In this example, if a company headquartered in New York is using GSLB, users in Tokyo can connect to a server closer to their location, resulting in a faster user experience.

GSLB to DNS

Previous article What is DNS? Introduction to Domain Name System – System Design 06 DNS is explained in depth, but in fact its operation is related to the concept of load balancing. DNS reorders the list of IP addresses in response to each DNS query, and different users will also get a list of IP locations. Users will also send requests to different servers to handle their requests, so DNS also distributes request traffic to different data centers through global server load balancing.

Local Load Balancing

Local load balancing is a load balancing limited to the data center, and the scope is limited to one data center. However, it is essentially the same as a load balancer, which allocates traffic to corresponding servers through different algorithms.

Load balancer implementation

The initial introduction mentioned that load balancers can be simply divided into hardware, software, and cloud load balancers. The following is an introduction to the implementation of the three:

Hardware load balancer

Load balancers have existed for 30 years, and load balancers were originally implemented in hardware. Of course, in the early days, it was very expensive to build a load balancer using hardware, but the advantage was that it could handle the user needs of the traffic at that time. However, in modern times, hardware load balancers are not necessarily the first choice for enterprises. The main disadvantage is that hardware load balancers are very dependent on hardware, require high maintenance and operating costs, and also need to deal with hardware compatibility issues.

Software load balancer

Compared with hardware, software load balancers have several additional features: high flexibility, programmability, and low cost. Software load balancers are also becoming more and more popular among enterprises because they are highly scalable and can be quickly set up as the system grows larger.

Cloud load balancer

Of course, in addition to software, with the emergence of the three major cloud service providers, load balancers can now also use cloud services. Among them, load balancer and service (LBaaS) also appeared at this time. Of course, if you are interested in other proper nouns, such as IaaS, PaaS, and SaaS, you can read this article Explain what IaaS & PaaS & SaaS are from Zeabur? Introduction to cloud service model!.

The biggest benefit of a cloud load balancer is that users can pay the cloud provider based on current usage. And compared with the technical threshold of hardware and software, the threshold of cloud load balancer is relatively low. Therefore, cloud load balancers are currently one of the main choices for many enterprises.

Conclusion

Load balancer is one of the important concepts in system design, and it is also an indispensable tool for large systems. In addition to improving system performance, it also ensures that the system will not crash due to high traffic in a short period of time.

If you are interested in other system designs, you may also wish to read other system design articles. You are also very welcome to leave messages for communication!

What is DNS? Introduction to Domain Name System – System Design 06

Introduction to System Design Components Building Block – System Design 05

Back-of-the-envelope Back-of-the-envelope Calculation – System Design 04

Non-functional features of software design – System Design 03

Application of abstraction in system design – System Design 02

Introduction to Modern System Design - System Design 01