Jinnianshilongnian.iteye.com/ open blog \
This article is excerpted from Nginx load balancing and Reverse proxy in chapter 2 of core Technologies of Web Architecture with 100 million traffic.
When a single instance of our application cannot support user requests, we need to expand from one server to two, dozens, hundreds. However, the user accesses the DNS server through www.jd.com. When requesting the DNS server, the browser first queries the DNS server to obtain the corresponding IP address, and then accesses the corresponding service through this IP address.
Therefore, one approach is to map www.jd.com domain names to multiple IP addresses. However, there is a simplest problem. If a server restarts or fails, DNS will have a certain cache time, take a long time to switch over after the failure, and there is no heartbeat check and retry mechanism for back-end services.
Therefore, extranet DNS should be used to implement GSLB (global load balancing) traffic scheduling, such as assigning users to the nearest server to improve the experience. And when the machine room in one area has a problem (such as a cut cable), DNS can be used to point to IP addresses in other areas to make the service available.
You can use “DNS query” in webmaster’s home. If you query C.3. cn, you can see a result similar to the following.
That is, different carriers return different public IP addresses.
For Intranet DNS, simple polling load balancing can be implemented. But, again, there is a certain amount of cache time and no retry mechanism. Therefore, we can consider options such as HaProxy and Nginx.
For general applications, Nginx is fine. However, Nginx is generally used for layer 7 load balancing, and its throughput is limited. In order to improve the overall throughput, an access layer will be introduced between DNS and Nginx. For example, LVS (software load balancer) and F5 (hard load balancer) can be used to perform four-layer load balancing, that is, first DNS resolution to LVS/F5, and then LVS/F5 forward to Nginx. Nginx forwards it to the back-end Real Server.
For the average business developer, we only need to care about the Nginx level, LVS/F5 is generally maintained by systems/operations engineers. Nginx currently provides HTTP (ngx_HTTP_upstream_module) layer 7 load balancing, and version 1.9.0 also supports TCP (ngx_STREAM_upstream_module) layer 4 load balancing.
Here are a few more concepts to clarify. Layer 2 load balancing rewrites the destination MAC address of packets to the MAC address of the upstream server. The source IP address and destination IP address remain the same. The load balancing server and real server share the same VIP mode, for example, LVS DR. Layer-4 LOAD balancing forwards packets to upstream servers (different IP addresses and ports) based on port numbers, such as LVS NAT mode and HaProxy. Layer-7 load balancing forwards packets to upstream servers (different IP addresses and ports) based on port numbers and host names and URLS of application layer protocols, such as HTTP. Such as HaProxy, Nginx.
LVS DR works at the data link layer. LVS and upstream servers share the same VIP. Load balancing is implemented by rewriting the destination MAC address of packets to the MAC address of the upstream server. However, LVS and upstream servers must reside on the same subnet. In order to solve cross-subnet problems without affecting load performance, HaProxy can be mounted behind LVS to solve cross-network and performance problems through HaProxy cluster of layer 4 to 7 load balancers. These two “half-baked” things complement each other and together form a “complete” load balancer. Now Nginx stream also supports TCP, so Nginx is a layer 4 to 7 load balancer and can be used to replace HaProxy with Nginx.
Before we move on, let’s get a few terms together. Access layer, reverse proxy server, load balancing server, and, unless otherwise specified in this article, Nginx. Upstream server (real server) : Upstream server to which Nginx loads loads.
There are several areas of concern for load balancing.
Upstream server configuration: Configure the upstream server using the upstream Server.
Load balancing algorithm: load balancing mechanism when multiple upstream servers are configured.
Retry failure mechanism: Configures whether to retry other upstream servers when timeout occurs or the upstream server does not survive.
Server heartbeat check: Health check or heartbeat check of upstream servers.
Nginx provides load balancing for upstream servers, such as load balancing, failover, failure retries, fault tolerance, health check, etc. When some upstream servers fail, requests can be forwarded to other upstream servers to ensure high availability, and intelligent load balancing can be achieved through OpenResty. For example, the hot spot and non-hot spot flow are separated, and the normal flow is separated from the crawler flow. The Nginx load balancer is also a reverse proxy server. The Nginx proxy sends user requests to an upstream server on the Intranet for processing. The reverse proxy server can cache and compress the response results to improve performance. The following figure shows Nginx as a load balancer/reverse proxy server.
This chapter begins with an introduction to Nginx HTTP load balancing and ends with an introduction to using Nginx to implement four-tier load balancing.
2.1 upstream configuration
The first step is to configure the upstream server for Nginx, i.e. the server to which the load is balanced. This is done by configuring upstream under the HTTP directive.
upstream backend {
Server 192.168.61.1:9080 weight = 1;
Server 192.168.61.1:9090 weight = 2;
}
Upstream Server main configuration.
IP address and Port: Set the IP address and port of the upstream server.
Weight: the weight used to configure the weight, the default is 1, the more weight high ration, the server the more requests (as above configuration for each request three times in 9080, forward a request to forward the remaining two requests to 9090), need according to the actual server processing power set weight (for example, physical server and virtual machine will require a different weight).
We can then configure the following proxy_pass to handle user requests.
location / {
proxy_pass http://backend;
}
When accessing Nginx, requests are reverse-proxy to Upstream Server configured in Backend. Let’s look at the load balancing algorithm.
2.2 Load balancing Algorithm
Load balancing resolves how to select Upstream Server to process incoming requests. By default, round-robin is used and several other algorithms are supported.
Round-robin: indicates round-robin. The default load balancing algorithm forwards requests to the upstream server in round-robin mode. Round-robin based on weights is implemented with the weight configuration.
Ip_hash: load balancing based on customer IP addresses. That is, load balancing is performed to the same Upstream Server using the same IP address.
upstream backend {
ip_hash;
Server 192.168.61.1:9080 weight = 1;
Server 192.168.61.1:9090 weight = 2;
}
Hash Key [consistent] : Hash a key or use the consistent hash algorithm for load balancing. The problem with using the Hash algorithm is that when one server is added/removed, many keys will be re-loaded to different servers (resulting in back-end problems). Therefore, it is recommended to consider using a consistent hashing algorithm so that when a server is added/removed, only a few keys will be re-loaded to different servers.
Hash algorithm: Here is load balancing based on the request URI. Nginx variables can be used. Therefore, complex algorithms can be implemented.
upstream backend {
hash $uri;
Server 192.168.61.1:9080 weight = 1;
Server 192.168.61.1:9090 weight = 2;
}
Consistent hash algorithm: consistent_key specifies this algorithm dynamically.
upstream nginx_local_server {
hash $consistent_key consistent;
Server 192.168.61.1:9080 weight = 1;
Server 192.168.61.1:9090 weight = 2;
}
The following location specifies a consistent hash key, where the parameter CAT (category) is given priority, and if not, load balancing is performed based on the request URI.
location / {
set $consistent_key $arg_cat;
if($consistent_key = “”) {
set $consistent_key $request_uri;
}
}
Instead, we set the consistent hash key through Lua.
set_by_lua_file $consistent_key”lua_balancing.lua”;
Lua_balancing. Lua code.
local consistent_key = args.cat
if not consistent_key or consistent_key == ” then
consistent_key = ngx_var.request_uri
end
local value = balancing_cache:get(consistent_key)
if not value then
success,err = balancing_cache:set(consistent_key, 1, 60)
else
newval,err = balancing_cache:incr(consistent_key, 1)
end
If the volume of a single class request is too large for the upstream server to handle, an incrementing count can be added to the consistent hash key to implement a poll-like algorithm.
if newval > 5000 then
consistent_key = consistent_key .. ‘_’.. newval
end
Least_conn: Loads requests to the upstream server with the fewest active connections. If fewer servers are configured, a weight-based polling algorithm will be used instead.
Nginx Business also provides LEast_time, which is load balancing based on minimum average response time.
2.3 Retry After Failure
There are two main configurations: upstream Server and proxy_pass.
upstream backend {
Server 192.168.61.1:9080 MAX_fails =2 Fail_timeout =10s weight=1;
Server 192.168.61.1:9090 Max_FAILS =2 Fail_timeout =10s weight=1;
}
Each upstream server is specified by configuring max_fails and FAIL_TIMEOUT for the upstream server. When max_FAILS requests for several times within fail_TIMEOUT, the upstream server is considered unavailable or not viable and will be removed. After fail_timeout, the server is added to the active upstream server list and retry.
location /test {
proxy_connect_timeout 5s;
proxy_read_timeout 5s;
proxy_send_timeout 5s;
proxy_next_upstreamerror timeout;
proxy_next_upstream_timeout 10s;
proxy_next_upstream_tries 2;
proxy_pass http://backend;
add_header upstream_addr $upstream_addr;
}
}
Then configure proxy_next_upstream and retry the next upstream server if an error is encountered.
Refer to the Nginx section of “Proxy Layer Timeout and Retry Mechanism” for detailed configuration.
2.4 Health Check
Nginx uses a lazy policy for health checks on upstream servers by default. Nginx Commercial edition provides health_check for active health checks. You can also integrate nginx_upstream_check_module (github.com/yaoweibin/n…
Nginx_upstream_check_module supports TCP and HTTP heartbeats for health checks.
2.4.1 Checking THE TCP Heartbeat
upstream backend {
Server 192.168.61.1:9080 weight = 1;
Server 192.168.61.1:9090 weight = 2;
check interval=3000 rise=1 fall=3 timeout=2000 type=tcp;
}
This section describes how to use TCP for heartbeat detection.
Interval: indicates the detection interval, which is configured every 3s.
Fall: How many times does the detection fail before the upstream server is identified as inviable.
Rise: The number of successful detections before the upstream server is identified as alive and ready to process the request.
Timeout: indicates the timeout period for detecting requests.
2.4.2 HTTP Heartbeat Check
upstream backend {
Server 192.168.61.1:9080 weight = 1;
Server 192.168.61.1:9090 weight = 2;
check interval=3000 rise=1 fall=3 timeout=2000 type=http;
Check_http_send “HEAD/HTTP / 1.0 RNRN status”.
check_http_expect_alive http_2xx http_3xx;
}
The following two additional configurations are required for HTTP heartbeat check.
Check_http_send: indicates the CONTENT of the HTTP request sent during the check.
Check_http_expect_alive: If the upstream server returns a matching response code, the upstream server is considered alive.
Ensure that the check interval is not too short. Otherwise, the upstream server may hang up due to too many heartbeat check packets. Set a proper timeout period.
This document uses openReSTY /1.11.2.1 (corresponding to nginx-1.11.2). Before installing nginx, you need to install the nginx_upstream_check_module patch (check_1.9.2+.patch). Go to the Nginx directory and execute the following shell:
Patch -p0 < /usr/servers/nginx_upstream_check_module-master/check_1.9.2+.patch.
If no patch is installed, the nginx_upstream_check_module module does not work. You are advised to use Wireshark to capture packets and check whether the nginx_upstream_check_module module works.
2.5 Other Configurations
2.5.1 Domain name Upstream Server
upstream backend {
Server c0.3. Cn;
Server c1.3. Cn;
}
Nginx community edition resolves domain names to IP addresses and logs them to upstream while Nginx is parsing configuration files. Upstream does not update when IP addresses corresponding to these two domain names change. Nginx business edition only supports dynamic updates.
However, proxy_pass c0.3.cn supports dynamic domain name resolution.
2.5.2 Backing up upstream Servers
upstream backend {
Server 192.168.61.1:9080 weight = 1;
Backup server 192.168.61.1:9090 weight = 2;
}
The upstream server of port 9090 is configured as the secondary upstream server. When all the primary upstream servers fail, requests are forwarded to the secondary upstream server.
For example, when the upstream servers are scaled down, some upstream servers need to be removed for the compression test, but some standby upstream servers are configured for insurance purposes. When all the upstream servers fail, traffic can be forwarded to the standby upstream server, so that user request processing is not affected.
2.5.3 Upstream Server Unavailable
upstream backend {
Server 192.168.61.1:9080 weight = 1;
Server 192.168.61.1:9090 weight = 2 down;
}
The upstream server of port 9090 is permanently unavailable. When the server is faulty or tested, the server can be removed temporarily through this configuration.
2.6 long connection
Configure long connections between Nginx and upstream servers. For long connections between clients and Nginx, refer to the corresponding section of “Timeouts and Retries” in the location.
The keepalive command configures the number of long connections.
upstream backend {
Server 192.168.61.1:9080 weight = 1;
Backup server 192.168.61.1:9090 weight = 2;
keepalive 100;
}
This directive configures the maximum number of cacheable idle connections between each Worker process and the upstream server. When this number is exceeded, the least recently used connection is closed. The Keepalive directive does not limit the total connection between Worker processes and upstream servers.
If you want to establish a long connection with the upstream server, do not forget the following configuration.
location / {
# support keep – the alive
Proxy_http_version 1.1;
proxy_set_header Connection “”;
proxy_pass http://backend;
}
If it is HTTP /1.0, you need to configure the sending of the “Connection: keep-alive” request header.
Do not forget to enable long connection support for upstream servers.
Next, let’s take a look at how Nginx implements keepalive (ngx_HTTP_upstream_keepalive _module) and get some code for connecting.
ngx_http_upstream_get_keepalive_peer(ngx_peer_connection_t*pc, void *data) {
/ / 1. First ask the load balancer which server (IP and port) to use
rc =kp->original_get_peer(pc, kp->data);
cache =&kp->conf->cache;
/ / 2. Polling “Free Connection Pool”
for (q =ngx_queue_head(cache);
q! = ngx_queue_sentinel(cache);
q =ngx_queue_next(q))
{
item = ngx_queue_data(q,ngx_http_upstream_keepalive_cache_t, queue);
c =item->connection;
/ / 2.1. If the connection IP addresses and ports cached by the Idle Connection Pool are the same as those to which the load is balanced, this connection is used
if (ngx_memn2cmp((u_char *)&item->sockaddr, (u_char *) pc->sockaddr,
item->socklen,pc->socklen)
= = 0)
{
/ / 2.2. Remove this connection from the Free Connection pool and push it to the top of the Free Connection Pool stack
ngx_queue_remove(q);
ngx_queue_insert_head(&kp->conf->free, q);
goto found;
}
}
/ / 3. If no long connections are available in the Free Connection Pool, a short connection is created
return NGX_OK;
Part of the code that releases the connection.
ngx_http_upstream_free_keepalive_peer(ngx_peer_connection_t*pc, void *data, ngx_uint_t state) {
c = pc->connection; // The current connection to release
/ / 1. If there are no pending connections in the “Free pool”, a space needs to be freed from the “free pool” for new connections (this situation occurs when the number of connections created exceeds the pool size, which can cause flapping)
if(ngx_queue_empty(&kp->conf->free)) {
q =ngx_queue_last(&kp->conf->cache);
ngx_queue_remove(q);
item= ngx_queue_data(q, ngx_http_upstream_keepalive_cache_t, queue);
ngx_http_upstream_keepalive_close(item->connection);
} else {/ / 2. Releases a connection from the Release Connection pool
q =ngx_queue_head(&kp->conf->free);
ngx_queue_remove(q);
item= ngx_queue_data(q, ngx_http_upstream_keepalive_cache_t, queue);
}
/ / 3. Push the current connection to the top of the Free Connection Pool stack for next use
ngx_queue_insert_head(&kp->conf->cache, q);
item->connection = c;
The total number of connections is the total number of long connections in the free connection pool + Free connection pool. First, the long connection configuration does not limit the total number of connections that the Worker process can open (the excess is considered as short connections). In addition, the connection pool must be properly set based on the actual scenario.
1. The free connection pool is too small and the connections are insufficient. You need to continuously build connections.
2. The idle connection pool is too large and too many idle connections are timed out before they are used.
In addition, it is recommended to enable long links only for tabloid articles.
2.7 Example of HTTP Reverse Proxy
In addition to implementing load balancing, reverse proxies also provide services such as caching to reduce stress on upstream servers.
1. Global Configuration (Proxy Cache)
proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 512 4k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 256k;
proxy_cache_lock on;
proxy_cache_lock_timeout 200ms;
proxy_temp_path /tmpfs/proxy_temp;
proxy_cache_path /tmpfs/proxy_cache levels=1:2keys_zone =cache:512m inactive=5m max_size=8g;
proxy_connect_timeout 3s;
proxy_read_timeout 5s;
proxy_send_timeout 5s;
If proxy buffer is enabled, the cache content will be stored in TMPFS (in-memory file system) to improve performance. Set the timeout period.
2. Location configuration
location ~ ^/backend/(.*)$ {
# set consistency hash load balancing key
set_by_lua_file $consistent_key “/export/App/c.3.cn/lua/lua_ balancing_backend.properties”;
Retry configuration on failure
proxy_next_upstream error timeout http_500 http_502 http_504;
proxy_next_upstream_timeout 2s;
proxy_next_upstream_tries 2;
# request the upstream server to use the GET method (regardless of the request method)
proxy_method GET;
Do not pass the request body to the upstream server
proxy_pass_request_body off;
Do not pass headers to upstream servers
proxy_pass_request_headers off;
Set which headers from the upstream server are not sent to the client
proxy_hide_header Vary;
# support keep – the alive
Proxy_http_version 1.1;
proxy_set_header Connection “”;
Pass Referer, Cookie, and Host to upstream server (pass on demand)
proxy_set_header Referer $http_referer;
proxy_set_header Cookie $http_cookie;
proxy_set_header Host web.c.3.local;
proxy_pass http://backend /$1$is_args$args;
}
Proxy_pass_request_body and proxy_PASS_request_HEADERS are enabled to disable the transmission of request headers and content bodies to the upstream server, so that the upstream server is not attacked by request headers and does not need to be parsed. If you need to pass, use proxy_set_header to pass as needed.
You can also enable GZIP support to reduce the size of packets transmitted over the network by doing the following.
gzip on;
gzip_min_length 1k;
gzip_buffers 16 16k;
Gzip_http_version 1.0;
gzip_proxied any;
gzip_comp_level 2;
gzip_types text/plainapplication/x-java text/css application/xml;
gzip_vary on;
For content-type responses, it is recommended to enable gzip compression. The gzip_comp_level compression level depends on the actual compression test (bandwidth versus throughput).