The following is just a reflection of one afternoon


What is load balancing

Application of load balancing in a distributed system, also can saying is a core module, main effect is to divide a large amount of homework properly to perform multiple operating units, is used to solve the high concurrency and high availability of the Internet architecture, using multiple with load balancing server components, instead of a single component, can improve the reliability through redundancy, Maximizing resource utilization, improving system throughput, response latency, etc., so I’m going to share this at the summary level, and I’m not going to go into too much detail on some of the basics, like load balancing patterns, algorithms, I’m not going to go into the details, there’s a lot of information out there on the web, I want to share my application and understanding of load balancing at a higher level.

What are the common load balancing methods

I’m talking about broad load balancing here, and I don’t specify Web-specific load balancing

Client mode

  1. HTTP uses DNS intelligent load balancing to resolve traffic to different servers based on domain names
  2. The built-in algorithm of the client automatically connects to multiple servers with multiple domain names
  3. Microservice architecture clients get a list of back-end services through the service registry (SpringCloud Stack && Kubkernetes Service && Dubbo, etc.)
  4. Sharding of mysql database and table clients
  5. Message queue Multiple consumer consuming tasks in the queue

From the above arrangement, we can see that there are more load balancing methods realized by using the client, which can be customized to a higher degree and have a greater degree of freedom. Change according to your own needs. The premise of doing this is to need a centralized storage, which can be the IP information related to DNS stored by DNS organizations. It can also be distributed storage like ETCD and ZooKeeper used in microservices and related information of Watch service. It can even be more simple like sharding of mysql client to directly unify all configuration files of client for one configuration, or even the data directly stored in memory.


Server mode

Server side load balancing is even more:

  1. Haproxy/LVS based on tcp4 layer
  2. Nginx, Trafik based on seven layers OF HTTP
  3. Related proxy mycat, Sharding-Sphere and Kingsharding based on layer 7 mysql
  4. Its software to achieve ElasticSearch, distributed storage

There are even direct hardware-based:

Hardware F5(– too expensive –)

As can be seen from the above, load balancing technology based on layer 4 and 7 will be relatively mature, there will be related products in various fields, and this layer is likely to be the traffic entry, will be some high-performance related software products, or even some expensive hardware products, but this layer is also the most interesting. You can do some more fun and useful features based on load balancing or load balancing.


More advanced applications of load balancing applications – load balancing has a lot to play with

7 layer HTTP

Since we have the access to the traffic and all the data, it is easy to customize the operation according to the data. I will give some examples, such as OpenResty, which is a high-performance Web platform based on Nginx and Lua. Because the kernel is still based on Nginx, the performance loss is very small. You can add your own custom functionality without changing the code, and since OpenResty is a 7-tier load balancer, you can get every HTTP connection, status code, URL,request parameter, and so on in the Lua code.


  1. Service gateway: Based on the connection, you can quickly design a program to prevent CC attacks, limit the IP access exceptions, and based on the URL, IP you can limit the number of calls per user per interface using algorithms such as token bucket traffic limiting.
  2. Custom scheduling algorithm: Based on the URL and request parameters, different traffic can be distributed to different server groups.
  3. Servized upstreaming: You can use a registry to hold the server of your backend services, then the Lua program goes to the Watch registry and dynamically updates the list of registrations.


Four layers of the TCP


The following is an example of layer 4. For example, based on LVS, layer 4 can obtain relatively less data, and it will be lower, and it will be more difficult to operate. Aliyun uses open source software Keeplived + LVS and tunnel VXLAN protocol to realize layer 4 load balancing based on multi-tenant.

A Ddos attack defense: Implements SYN flood attacks based on the SYNPROXY module

B LVS cluster implementation: THE LVS and connected switches run OSPF, and the LVS and switches run OSPE heartbeat. When one or more LVS are down, other LVS clusters provide services.

C Tenant isolation: The LVS cluster and back-end multi-tenants use vxLAN for traffic, ensuring LVS isolation in cloud load balancing.

Describe a software based:


I believe that larger companies will use ElasticSearch as a search engine, as well as its clustering model. I won’t go into details about the implementation of elasticSearch. If you are interested, you can check out the official website.

Elasticsearch uses the concept of load balancing to implement data fragmentation (data load balancing), request forwarding (computational load balancing)

  1. Data sharding: ElasticSearch implements index sharding itself. First of all, it is used to load balance data. Data is distributed and not concentrated on a single node to ensure high availability of colleagues.
  2. Data synchronization: Multiple copies of data sharding are used to ensure high availability and load balancing of read data. During data query, the read data will be dispersed into separate data fragments. If there are multiple copies, the data can be randomly selected from the multiple copies for query to improve query performance.

You can certainly learn from the sharded copy strategy above, and apply it to things like distributed storage and database design.

For ElasticSearch, there are a variety of load balancing algorithms. You can use them based on CPU/MEM/NETWORK IO, fragment number, index number, and even custom algorithms. This is very much like the openstack VM scheduling algorithm and kubernetes Pods scheduling algorithm.

Simple described above a few of the applications based on load balancing, load balancing technology in the Internet really is pretty basic, use face is also very wide, like a database, a message queue, caching technology such as using the concept of load balancing, in general, is whether we in software design program, or system design, structure optimization, Think about whether you can use load balancing to solve your problem.

Let me summarize:

To put it simply, load balancing is to consider the client side and the server side respectively, and to disperse computing tasks, data nodes and network tasks, so as to improve the availability and scalability. What I said here is rather empty. I just need to choose appropriate scheduling policies according to the actual situation and solve my own problems.