Origin of Load Balancing
At the beginning of the business, we usually use a single server to provide external services. With the increasing traffic, no matter how optimized a single server is, no matter how good the hardware is, there will always be a performance ceiling. When the performance of a single server cannot meet the business needs, it is necessary to form a cluster system of multiple servers to improve the overall processing performance.
Based on the above requirements, we need to use a unified traffic entrance to provide services externally. In essence, a traffic scheduler is required to distribute a large number of user requests to different servers in the cluster in a balanced manner through a balanced algorithm. That’s what we’re talking about today in terms of load balancing.
There are several benefits to using load balancing:
- Improve the overall performance of the system;
- The expansibility of the system is improved.
- Improved system availability;
Load Balancing Type
Broadly, load balancers can be classified into three types: DNS load balancers, hardware load balancers, and software load balancers.
(1) DNS implements load balancing
DNS is the simplest way to implement load balancing. A domain name is resolved to multiple IP addresses by DNS, and each IP address corresponds to a different server instance. In this way, traffic is scheduled. A simple load balancing function is implemented without using a common load balancer.
DNS load balancing has the advantages of simple implementation and low cost. There is no need to develop or maintain load balancing devices. However, it has some disadvantages:
-
The server failover delay is large and the server upgrade is not convenient
. We know that the DNS and users are caches at different levels. Even if the faulty server is modified or removed by DNS in time when a fault occurs, the DNS cache of carriers may not follow the TTL rule, resulting in the DNS taking effect very slowly. Sometimes, there will be some request traffic a day later.
-
The traffic scheduling is unbalanced and the granularity is too coarse
. The balance of DNS scheduling depends on the IP address list returned by local carriers. Some carriers do not return multiple IP addresses in polling. In addition, the number of users that a certain operator serves behind LocalDNS will also constitute an important factor of uneven traffic scheduling.
-
The traffic allocation policy is too simple and supports too few algorithms
. DNS generally supports only RR polling and simple traffic allocation policies, but does not support scheduling algorithms such as weights and Hash.
-
The IP list supported by DNS is limited
. As we know, DNS uses UDP packets to transmit information, and the size of each UDP packet is limited by the MTU of the link. Therefore, the number of IP addresses stored in packets is very limited. Ali DNS system supports 10 different IP addresses for the same domain name.
In fact, this approach to load balancing is rarely used in production environments, since the disadvantages are obvious. DNS load balancing is described in this document to explain the concept of load balancing more clearly.
Companies with the size of BAT generally use DNS to achieve global load balancing at the geographical level to realize nearby access and improve access speed. This method is generally based on load balancing of inbound traffic, and the load architecture is realized by more professional load balancing equipment at the lower level.
(2) Hardware load balancing
Hardware load balancers are dedicated hardware devices to implement load balancing functions. Currently, there are two typical hardware load balancers: F5 and A10.
This kind of equipment has strong performance and powerful functions, but the price is very expensive. Generally, only tuhao companies will use this kind of equipment, and small and medium-sized companies generally cannot afford it. The business volume is not so large, and it is quite wasteful to use these equipment.
Advantages of hardware load balancing:
- Powerful: supports load balancing at all levels and load balancing algorithms.
- Powerful performance: Performance far exceeds that of common software load balancers.
- High stability: Commercial hardware load balancers have been rigorously tested and are highly stable after extensive use.
- Security protection: Provides security functions such as firewall and anti-ddos attack, and supports SNAT.
The disadvantages of hardware load balancing are also obvious:
- The price expensive;
- Poor scalability, unable to be extended and customized;
- Debugging and maintenance is more troublesome, need professional personnel;
(3) Software load balancing
Software LOAD balancing software can be used to run load balancing software on common servers to implement load balancing. At present, common Nginx, HAproxy, LVS. The differences:
- Nginx: 7-layer load balancing, supports HTTP, E-mail protocols, and supports 4-layer load balancing.
- HAproxy: supports seven layer rules, performance is also very good. The default load balancing software used by OpenStack is HAproxy.
- LVS: runs in kernel mode and has the highest performance among software load balancers. Strictly speaking, it works at layer 3, so it is more general and suitable for various application services.
Advantages of software load balancing:
- Easy to operate: Relatively simple to deploy and maintain;
- Cheap: the software is free for the cost of the server;
- Flexible: Layer-4 and layer-7 load balancers can be selected based on service characteristics, facilitating expansion and customization.
Load balancing LVS
Software load balancing mainly includes Nginx, HAproxy and LVS, which are commonly used. LVS is basically used in the four-layer load balancing. It is known that BAT and other large factories are heavy users of LVS because of the excellent performance of LVS, which can save huge costs for the company.
LVS, full name Linux Virtual Server, is an open source project initiated by Dr. Zhang Wensong from China. It has great popularity in the community. It is a reverse proxy Server based on four layers with strong performance.
It is now part of the standard kernel and features reliability, high performance, scalability, and operability to achieve optimal performance at a low cost.
Basic principles of Netfilter
LVS is a load balancing function implemented by the NetFilter framework in the Linux kernel. Therefore, you must understand the basic working principles of NetFilter before learning LVS. Netfilter is a very complex tool. Linux firewall is called NetFilter, but iptables is used to write and transfer rules in user space. The following figure shows the working mechanism of NetFilter:
Netfilter is a kernel Linux firewall mechanism. As a general and abstract framework, netFilter provides a set of hook function management mechanism, providing functions such as packet filtering, network address translation, and connection tracing based on protocol type.
In plain English, NetFilter provides a mechanism to set up several levels (hook functions) according to rules to perform related operations during the packet flow. Netfilter sets a total of five points, including PREROUTING, INPUT, FORWARD, OUTPUT, and POSTROUTING
- PREROUTING: Packets that have just entered the network layer but have not been searched for routes
- INPUT: Determines the packet sent to the local host through route search
- FORWARD: indicates the packet to be forwarded after the route search, before POST_ROUTING
- OUTPUT: The packet just sent from the native process through here
- POSTROUTING: Enter the network layer for the packet that is about to leave the device after route searching and forwarding
When a packet enters the network card, passes through the link layer and then enters the network layer PREROUTING occurs, which is followed by route look-up based on the destination IP address, if the destination IP address is local, the packet continues on to the INPUT, and passes through the protocol stack to the appropriate application based on the port.
The application processes the request and sends the response packet to OUTPUT, which is finally sent out to the network card after POSTROUTING.
If the destination IP is not local and the server has the forward parameter turned on, the packet will be delivered to the forward and finally sent out to the network card after POSTROUTING.
LVS fundamentals
LVS is based on the NetFilter framework and mainly works on the INPUT chain. Ip_vs_inHOOK function is registered on the INPUT to carry out the main process of IPVS, as shown in the figure below:
- When a user accesses www.sina.com.cn, the user data passes through the layers of the network, and finally through the switch into the LVS server network card, and into the kernel network layer.
- After PREROUTING, route lookup is performed to determine that the destination VIP is the local IP address, so packets enter the INPUT chain
- LVS works on the INPUT chain, and will determine whether the request is LVS service according to the IP:Port accessed. If so, the MAIN LVS process will be carried out, relevant data of the data package will be forcibly modified, and the data package will be sent to POSTROUTING chain.
- After receiving the data packet on POSTROUTING, the data packet is sent to the back-end server based on the destination IP address (back-end real server) through route selection.
The open source LVS version has three working modes, each of which has its own advantages and disadvantages and different application scenarios, including the following three modes:
- DR model
- NAT mode
- Tunnel model
Another schema that must be mentioned here is FullNAT, which is not available in the open source version. This model was first originated in Baidu, and then carried forward in Ali. It is open source by Ali team, and the code address is as follows:
- github.com/alibaba/lvs
The LVS website also has a download address, but it is not integrated into the kernel mainline version.
The FullNAT mode will be covered in a section later. The following describes the DR, NAT, and Tunnel modes in detail.
Implementation principle of DR mode
The LVS schematic is relatively simple and represents a more general process. The following describes how the DR mode works in detail based on the implementation principle of the DR mode.
Actually DR is the most commonly used working mode because of its powerful performance. The following attempts to describe how the DR pattern works in terms of a request and response data flow
(1) Implementation principle process
① When the client requests www.sina.com.cn home page, the request packet travels across the network to Sina’s LVS server nic: the source IP is the client IP address CIP, the destination IP is Sina’s external server IP address, namely VIP; In this case, the source MAC address is CMAC, which is actually the MAC address of the router connected to the LVS (CMAC for easy understanding), and the target MAC address is the MAC address corresponding to the VIP, which is VMAC.
(2) Packets go through the link layer and arrive at PREROUTING position (just entering the network layer). After route lookup, the destination IP address is FOUND to be LVS VIP, and the packets are sent to the INPUT chain. At this time, the MAC, IP, and Port of the packets are not changed.
③ The packet arrives at the INPUT chain, where LVS does its main work. In this case, LVS will check whether it is the SERVICE defined by LVS according to the destination IP address and Port. If it is a defined VIP service, LVS will select one from the list of real servers as RS1 according to the configuration information, and then search the route in the Out direction with RS1 as the target. Determine the hop information and through which network card the packet is sent. Finally, the packet is delivered to the OUTPUT chain.
(4) After the packet passes the POSTROUTING chain, it is transferred from the network layer to the link layer, and the destination MAC address is changed to the MAC address of the RealServer server, which is marked as RMAC. The source MAC address is changed to the MAC address corresponding to selfIP on the same network segment as LVS and RS, denoted as DMAC. At this point, the packet is forwarded through the switch to the RealServer server (note:
Switches are not drawn for simplicity
).
⑤ After the request packet reaches the back-end real server, the link layer checks that the destination MAC address is its nic address. At the network layer, the route is searched and the destination IP address is VIP (VIP is configured on LO). It is determined that the packet is from the local host and copied to the application program (such as nginx server) through the protocol stack. After nginx responds to the request, the response packet is generated.
Then use CIP to find out the route, determine the next hop information and send network card device information. In this case, the source AND destination IP addresses of packets are VIP and CIP respectively, and the source MAC address is RMAC of RS1. The destination MAC address is the MAC address of the next hop (router), denoted as CMAC (CMAC for easy understanding). Then the data packet is forwarded to the real client through the router connected with RS to complete the whole process of request and response.
It can be seen from the whole process that the LOGIC of LVS in DR mode is relatively simple. The packets are forwarded to the back-end server through direct routing, and the response packets are sent directly from the RS server to the client without going through LVS.
As we know, the request packet is usually small and the response packet is large, and the packets passing through LVS are basically small packets. Therefore, this is the main reason for the strong performance of DR mode of LVS.
(2) Advantages and disadvantages and application scenarios
- Advantages of DR mode
The response data does not pass through LVS, providing high performance
Small modification of packets, complete information preservation (with client source IP)
- Disadvantages of DR mode
LVS and RS must be on the same physical network (cross-room is not supported)
Lo and other kernel parameters must be configured on the server
Port mapping is not supported
- Application scenarios of DR mode
If performance requirements are high, the DR mode can be selected and the source IP address of the client can be transparently transmitted.
Implementation principles of NAT mode
The second working mode of LVS is NAT mode. The following figure details the whole process of data packets entering LVS from clients and forwarding them to RS. Then RS forwards the response data to LVS again, and LVS replies the data packets to clients.
(1) Implementation principle and process
① The user request packet passes through layers of networks and reaches the LVS network card. At this time, the source IP of the packet is CIP and the destination IP is VIP.
(2) Enter the prerouting position at the network layer through the NETWORK adapter, search for routes based on the destination IP address, confirm that the packets are from the local IP address, and forward the packets to the INPUT. In this case, the source IP address and destination IP address remain unchanged.
3. After arriving at the LVS, check whether the IPVS service is enabled based on the destination IP address and destination port. If the IPVS service is used, an RS server is selected as the back-end server, the destination IP address of the packet is changed to RIP, and RIP is used as the destination IP address to search for routing information, determine the next hop and egress information, and forward the packet to output.
(4) After postrouting and link layer processing, the modified data packet arrives at the RS server. At this time, the source IP address of the data packet is CIP and the destination IP address is RIP.
⑤ The data packets arriving at the RS server are sent to the user-space Nginx program after being checked by the link layer and network layer. When the nginx program finishes processing, it sends a response packet. Since the default gateway on RS is configured as the IP address of the LVS device, the Nginx server forwards the packet to the next hop, namely the LVS server. In this case, the source IP address of the packet is RIP and the destination IP address is CIP.
⑥ After receiving the RS response packet, the LVS server searches the route and finds that the destination IP address is not the local IP address, and the LVS server has enabled the forwarding mode, so the packet is forwarded to the forward chain, and the packet is not modified.
⑦ After receiving the response packet, LVS will search the service and connection table according to the destination IP and destination port, change the source IP to VIP, determine the next hop and exit information through the route search, send the packet to the gateway, through the complex network to reach the user client, and finally complete a request and response interaction.
In NAT mode, bidirectional traffic passes through LVS, causing performance bottlenecks in NAT mode. However, NAT differs from other modes in that it supports port mapping and the Windows operating system.
(2) Advantages, disadvantages and application scenarios
- Advantages of NAT mode
Supports Windows operating system
Port mapping is supported. If the RS port is inconsistent with the Vport, the LVS modifies the Dport as well as the destination IP address to support port mapping.
- Disadvantages of NAT mode
The back-end RS needs to be configured with a gateway
The bidirectional flow has a large load pressure on LVS
- Application scenarios of the NAT mode
If you are running Windows and using LVS, you must choose NAT mode.
Implementation principle of Tunnel mode
Tunnel mode is rarely used in China, but Tencent is said to use a lot of Tunnel mode. It is also a one-arm mode, only the request data will go through LVS, and the response data will be sent directly from the backend server to the client. It is also very powerful, and supports cross-room. Below continue to look at the diagram analysis principle.
(1) Implementation principle and process
① The user request packet passes through the multi-layer network and arrives at the LVS network card. At this time, the source IP address of the packet is CIP and the destination IP address is VIP.
(2) Enter the prerouting position at the network layer through the network adapter, search for routes based on the destination IP address, confirm that it is the local IP address, forward the packets to the INPUT chain, and arrive at the LVS. At this time, the source IP address and destination IP address remain unchanged.
3. After arriving at the LVS, check whether the IPVS service is enabled based on the destination IP address and destination port. If the IPVS service is used, it selects an RS server as the back-end server, searches for routing information using rip as the destination IP address, determines the next hop and dev information, and adds an IP header (rip as the destination IP address and DIP as the source IP address) to the IP header. Forward the packet to output.
④ Data packets are sent to the router gateway through the LVS network card according to the routing information, and then to the back-end server through the network.
⑤ After the back-end server receives the packet, the IPIP module unloads the header of the Tunnel. Normally, the source IP is CIP and the destination IP is VIP. Because VIP is configured on TUNl0, the IP is determined as the local IP after route search and sent to the application program. The application nginx responds to the data normally with VIP as the source IP and CIP as the destination IP packets sent out of the network card and finally to the client.
Tunnel mode has the high performance of DR mode and supports cross-room access. It sounds perfect. However, domestic carriers have certain characteristics. For example, the source IP address of RS response packet is VIP, and VIP and back-end server may cross carriers, and may be blocked by carriers’ policies. Tunnel has not been used in production environment, so it may be difficult to implement Tunnel in China.
(2) Advantages, disadvantages and application scenarios
- Advantages of Tunnel mode
Single arm mode, low pressure on LVS load
The data packet modification is small, and the information is saved completely
Cross-machine room (but it is difficult to achieve in China)
- Disadvantages of Tunnel mode
The IPIP module needs to be installed and configured on the back-end server
VIP needs to be configured on back-end server TUNl0
Adding a tunnel header may lead to fragmentation, which affects server performance
The IP address of the tunnel header is fixed, and the hash of the nics of the back-end server may be uneven
Port mapping is not supported
- Scenarios of Tunnel mode
In theory, if the forwarding performance is high and cross-room requirements are required, the Tunnel may be the best choice.
So far, THE PRINCIPLE of LVS has been clearly explained, the content is more, it is recommended to read twice, because the length of the article is too long, the content of practical operation will be put into the next article.
Three things to watch ❤️
If you find this article helpful, I’d like to invite you to do three small favors for me:
-
Like, forward, have your “like and comment”, is the motivation of my creation.
-
Follow the public account “Java rotten pigskin” and share original knowledge from time to time.
-
Also look forward to the follow-up article ing🚀
-
[666] Scan the code to obtain the learning materials package
