One of the high concurrency solutions – load balancing

Click on the “Python Guide” at the top to get a full set of Python videos.

Technical articles first time delivery!

1. What is load balancing?

When the performance of a single server reaches its limit, we can use a cluster of servers to improve the overall performance of the site. Therefore, in a server cluster, one server is required to act as the scheduler, which receives all user requests first, and then assigns the requests to a back-end server for processing according to the load of each server.

In this process, how can the scheduler reasonably allocate tasks to ensure that all back-end servers give full play to their performance, so as to keep the overall performance of the server cluster optimal? This is the load balancing problem.

The following describes the four implementation modes of load balancing.

(a) HTTP redirection to achieve load balancing

Process description

When a user sends a request to the server, the request is first intercepted by the cluster scheduler. The dispatcher selects a server according to a certain allocation policy, encapsulates the IP address of the selected server in the Location field of the HTTP response message header, sets the status code of the response message to 302, and finally returns the response message to the browser.

When the browser receives the response message, it parses the Location field and makes a request to the URL, which is then processed by the specified server and returned to the user.

In the process of using HTTP redirection for server cluster load balancing, one server is required as the request scheduler. An operation requires two HTTP requests: one to the scheduling server to obtain the IP address of the back-end server, and the second to the back-end server to obtain the processing result.

Scheduling policy

After receiving a request from a user, the scheduling server selects which back-end server to process the request based on the scheduling policy used by the scheduling server.

1. Random allocation Policy After receiving a user request, the scheduling server randomly decides which back-end server to use, encapsulates the IP address of the server in the Location attribute of the HTTP response message, and returns it to the browser.

2. The Polling policy (RR) Scheduling server needs to maintain a value to record the IP address of the last allocated back-end server. When new requests arrive, the scheduler allocates them to the next server in turn.

Because the polling policy requires the scheduler to maintain a value to record the last assigned server IP, it incurs additional overhead; In addition, because this value is a mutex resource, when multiple requests come in at the same time, the mutex resource needs to be locked to avoid thread safety issues, which reduces performance. A random allocation strategy does not require additional values to be maintained, and thus does not have thread-safety issues, and therefore performs better than polling.

Advantages and disadvantages analysis

Using HTTP redirection to realize the load balancing of server cluster is easy to realize, the logic is relatively simple, but the disadvantages are also more obvious.

In HTTP redirection, the scheduling server only comes into play when the client first makes a request to the web site. After the scheduling server returns the response information to the browser, the subsequent operations of the client are based on the new URL (that is, the back-end server). After that, the browser will not have a relationship with the scheduling server, which may cause the following problems:

Because different users have different access times and different page depths, each user exerts different stress on their own back-end servers. However, the scheduling server has no way to know how much pressure the current user will cause to the server during the scheduling, so this way can not realize the true load balancing, but is to evenly distribute the number of requests to each server. If the back-end server assigned to the user fails, and if the page is cached by the browser, the request will be sent to the failed server when the user accesses the site again, resulting in an access failure.

(2) DNS load balancing

What is DNS?

Before learning about DNS load balancing, you need to learn about DNS domain name resolution.

We know that packets of data use IP addresses to travel around the network, and to facilitate user memory, we use domain names to visit websites. So, before we can access a website through a domain name, we first need to resolve the domain name into an IP address, which is done by DNS. That’s the domain name server.

The requests we submit are not sent directly to the site we want to visit, but first to the domain name server, which resolves the domain name into an IP address and returns it to us. We don’t make a request to an IP until we receive it.

Therefore, the DNS server has a natural advantage. If a domain name refers to multiple IP addresses, the DNS only needs to select one IP address to return to users during domain name resolution to implement load balancing in the server cluster.

The specific practices

First, we need to point our domain name to multiple back-end servers (resolve a domain name to multiple IP addresses), then set up a scheduling policy, and then we are ready to complete the load balancing by the DNS server.

When a user initiates a request to our domain name, the DNS server automatically selects an appropriate IP address according to the preset scheduling policy, and then the user initiates a request to this IP address.

Scheduling policy

Generally, DNS providers provide scheduling strategies for us to choose, such as random allocation, polling, and assigning the server closest to the requester based on his geography.

Advantages and disadvantages analysis

The advantage of DNS load balancing is that it is easy to configure. DNS servers are responsible for scheduling the cluster of servers, so we can focus on the back-end servers to ensure their stability and throughput. And you don’t have to worry about DNS server performance, even with polling, its throughput is still excellent.

In addition, DNS load balancing is more scalable, you can resolve many IP addresses for a domain name without worrying about performance.

However, by handing over cluster scheduling to the DNS server, we can’t control the scheduler and customize the scheduling policy.

The DNS server cannot know the load of each server, so it cannot implement load balancing in the true sense. It is the same as HTTP redirection, except that all requests are equally distributed to the back-end server.

In addition, when we find a back-end server failure, even if we immediately remove the server from domain name resolution, the DNS server will have cache, the IP will still remain in DNS for a period of time, so that some users can not access the website properly. This is a fatal question! Fortunately, this problem can be solved with dynamic DNS.

Dynamic DNS

Dynamic DNS allows us to programmatically change domain name resolution in the DNS server. So that when our monitor detects that a server is down, it can immediately notify DNS to remove it.

From what has been discussed above

DNS load balancing is a crude load balancing method. It is only introduced here and not recommended.

(3) Reverse proxy load balancing

What is reverse proxy load balancing?

The reverse proxy server is a server located in front of the actual server. All requests sent to our website first go through the reverse proxy server. The server either directly returns the results to the user according to the user’s request, or sends the request to the back-end server for processing, and then returns it to the user.

Earlier we described caching static and commonly used dynamic pages using a reverse proxy server. Next, let’s look at the more common functionality of reverse proxy servers — load balancing.

We know that all requests sent to our site first go through a reverse proxy server. The reverse proxy server can then act as a dispatcher for a cluster of servers, forwarding requests to an appropriate server based on the current back-end server load and returning the processing results to the user.

advantages

1. Hide the back-end server. In contrast to HTTP redirects, reverse proxies can hide backend servers from all browsers, ensuring control of the scheduler and improving overall cluster performance.

2. Failover. Compared with DNS load balancing, reverse proxy can remove faulty nodes more quickly. When the monitor detects a back-end server failure, it notifies the reverse proxy server and deletes it immediately.

3. Assign tasks Properly HTTP redirection and DNS load balancing fail to implement load balancing, that is, the scheduling server cannot assign tasks based on the actual load of back-end servers. However, the reverse proxy server supports manually setting the weight of each back-end server. We can set different weights according to the configuration of the server. Different weights will lead to different probabilities of being selected by the scheduler.

disadvantages

The reverse proxy server processes all requests first. If the number of requests exceeds the maximum load of the scheduling server, the throughput of the scheduling server deteriorates the overall cluster performance.

When back-end servers cannot meet the huge throughput, the number of back-end servers needs to be increased, but it cannot be increased indefinitely because it is constrained by the maximum throughput of the scheduling server.

Viscous session

Reverse proxy servers can cause a problem. If a back-end server processes a user’s request and stores the user’s session or cache, then when the user sends a request again, there is no guarantee that the request will still be processed by the server that holds the user’s session or cache. If it is processed by another server, the previous session or cache will not be found.

Solution 1: Modify the task assignment policy of the reverse proxy server to use the user IP address as the identifier. The same user IP is processed by the same back-end server, thus avoiding the problem of sticky sessions.

Solution 2: The server ID of the request can be marked in the Cookie. When the request is submitted again, the scheduler can assign the request to the server marked in the Cookie.

2. Load balancing components

apache

— Apache Software Foundation is an open source cross-platform web server, belonging to the old web server, support based on Ip or domain name virtual host, support proxy server, support secure Socket layer (SSL) and so on, the Internet is mainly used to do static resource server, Can also do proxy server forwarding requests (such as: picture chain, etc.), combined with Tomcat and other servlet container processing JSP.

ngnix

A high-performance HTTP and reverse proxy server developed by the Russians. Because Nginx exceeds Apache’s high performance and stability, there are more and more websites using Nginx as a Web server in China, including Sina blog, Sina podcast, netease news, Tencent, Sohu blog and other portal channels. In a high concurrency environment of more than 3W, Ngnix has 10 times the processing power of Apache. Reference: apache tomcat and performance analysis and comparison (http://blog.s135.com/nginx_php_v6/)

lvs

— Short for Linux Virtual Server, Linux Virtual Server is a Virtual Server cluster system. Founded in May 1998 by Dr. Zhang Wensong who graduated from National University of Defense Technology, it can realize simple load balancing under LINUX platform. Learn more, visit the website: http://zh.linuxvirtualserver.org/.

HAProxy

HAProxy provides high availability, load balancing, and proxy based on TCP and HTTP applications. It supports virtual hosting. It is a free, fast, and reliable solution. HAProxy is especially useful for heavily loaded Web sites that require session persistence or seven-tier processing. HAProxy runs on current hardware and can support tens of thousands of concurrent connections. And its operating mode makes it easy and secure to integrate into your current architecture, while protecting your Web server from being exposed to the network.

keepalived

Keepalived is not a property field on a component like Apache or Tomcat. Keepalived is a component that makes web servers availably HA. It can detect the working status of the Web server. If the fault of the server is detected, it is removed from the server group. After the server works properly, Keepalive automatically detects and adds it to the server group. Achieve instantaneous seamless IP exchange when the active and standby servers fail. It is a user-space daemon for LVS cluster node health detection and a director failover module for LVS. The Keepalived daemon can check the state of the LVS pool. If a server in the LVS server pool goes down. Keepalived notifies the kernel to remove the node from the LVS topology with a setsockopt call.

memcached

It is a high performance distributed memory object caching system. Originally developed by Danga Interactive for LiveJournal’s rapid growth, the system was used to cache business query data and reduce database load. Its daemons are written in C, but the client supports almost all languages (there are basically three versions of the client (memcache client for Java; Spymemcached;xMecache)), and the server communicates with the client through a simple protocol. The data cached in memcached must be serialized.

terracotta

– is a famous open source Java cluster platform developed by Terracotta. It implements an abstraction layer between the JVM and Java applications that deals specifically with clustering capabilities, allowing users to implement clustering of Java applications without changing system code. Data persistence, session replication, and high availability (HA) are supported.

Author: Vulcanhy_IT

Links:

https://blog.csdn.net/github_37515779/article/details/79953788