An incomplete guide to OpenResty

The author | Mr Wong chiu

Almond operation and maintenance engineer, focusing on container technology and automatic operation and maintenance.

OpenResty profile

OpenResty® is a high-performance Web platform based on Nginx and Lua. We know that the development of Nginx modules need to use C language, but also familiar with its source code, the cost and threshold is relatively high. LuaJIT VM embedded in Nginx, so that you can directly program on Nginx through Lua scripts, but also provides a large number of class libraries (such as: Lua-resty-mysql lua-resty-redis, etc.), which directly extends a Nginx Web Server into a Web framework, thanks to the high performance of Nginx, It can quickly construct a high-performance Web application system capable of concurrent connection of single machine over 10K or even 1000K.

Nginx adopts the master-worker model, where a master process manages multiple worker processes. Workers are really responsible for processing client requests, while master is only responsible for some global initialization and worker management. In OpenResty, each worker has a Lua VM. When a request is assigned to the worker, a Coroutine is created in the Lua VM of the worker to handle it. Data isolation between coroutines, each with an independent global variable _G.

OpenResty handles the request flow

Because Nginx divides a request into stages, third-party modules can mount it to different stages depending on their behavior. OpenResty applies the same features. It is a feature of OpenResty that different stages have different processing behaviors. The process for OpenResty to process a Request is shown below (starting with Request Start) :

instruction	Using range	explain
int_by_lua*init_worker_by_lua*	http	Initialize global configuration/preload Lua modules
set_by_lua*	server,server if,location,location if	Set the nginx variable, which is blocked. Lua code has to be very fast
rewrite_by_lua*	http,server,location,location if	Rewrite stage processing allows for complex forward/redirect logic
access_by_lua*	http,server,location,location if	Request access phase processing for access control
content_by_lua*	location, location if	A content handler that receives requests, processes them, and outputs responses
header_filter_by_lua*	HTTP server location location if	Set heade and cookie
body_filter_by_lua*	HTTP server location location if	Filter the response data, such as truncation and replacement
log_by_lua	HTTP server location location if	Log phase processing, such as logging visits/average response time statistics

Please refer to the official documentation for more details

Configuration OpenResty

The Lua code for OpenResty is included in the nginx.conf configuration file and can be written with the configuration file or loaded in a file:

Inline in nginx.conf:

server { ... location /lua_content { # MIME type determined by default_type: default_type 'text/plain'; content_by_lua_block { ngx.say('Hello,world! ')}}... }Copy the code

By loading a Lua script:

server { ... location = /mixed { rewrite_by_lua_file /path/to/rewrite.lua; access_by_lua_file /path/to/access.lua; content_by_lua_file /path/to/content.lua; }... }Copy the code

Shared scope of OpenResty variables

The global variable

In OpenResty, true global variables can only be defined at the init_by_lua* and init_worker_by_lua* stages. At other stages, OpenResty sets up an isolated global change table so as not to contaminate other requests in the process. Even if you can define global variables in the above two phases, avoid doing so. Problems solved by global variables can also be solved by module variables, and will be clearer and cleaner.

Module variable

Variables defined in Lua modules are referred to here as module variables. Lua VM will replace the required module with a package.loaded table, and the variables in the module will be cached. Within the same Lua VM, the variables in the module will be shared with each request, so that global variables can be avoided. Here’s an example:

nginx.conf

worker_processes 1; . location { ... lua_code_cache on; default_type "text/html"; content_by_lua_file 'lua/test_module_1.lua' }Copy the code

lua/test_module_1.lua

local module1 = require("module1")

module1.hello()Copy the code

lua/module1.lua

local count = 0
local function hello() 
    count = count + 1
    ngx.say("count: ", count)
end

local _M  = {
    hello = hello
}   

return _MCopy the code

When accessed through a browser, you can see that the count output is incremented, which also indicates that the module variables in Lua /module1. Lua are shared on each request:

count: 1
count: 2
.....Copy the code

Also, if the number of Worker_processes is greater than one, the result might be different. Because there is a Lua VM in each worker, module variables are only in the same VM, and all requests are shared. If sharing across Worker processes consider using ngx.shared.DICT or Redis for example.

The local variable

In contrast to global and module variables, we will refer to variables defined in *_by_lua* as local variables. Local variables are only valid for the current phase. If you want to use them across phases, use ngx.ctx or attach them to module variables.

Here we use the ngx. CTX table to pass the variable foo in three different stages:

location /test {
     rewrite_by_lua_block {
         ngx.ctx.foo = 76
     }
     access_by_lua_block {
         ngx.ctx.foo = ngx.ctx.foo + 3
     }
     content_by_lua_block {
         ngx.say(ngx.ctx.foo)
     }
 }Copy the code

Note, in addition, that each request, including subrequests, has its own ngX.ctx table. Such as:

 location /sub {
     content_by_lua_block {
         ngx.say("sub pre: ", ngx.ctx.blah)
         ngx.ctx.blah = 32
         ngx.say("sub post: ", ngx.ctx.blah)
     }
 }

 location /main {
     content_by_lua_block {
         ngx.ctx.blah = 73
         ngx.say("main pre: ", ngx.ctx.blah)
         local res = ngx.location.capture("/sub")
         ngx.print(res.body)
         ngx.say("main post: ", ngx.ctx.blah)
     }
 }Copy the code

Access GET /main output:

$pre sub Post: 32 main post: 73Copy the code

Performance switch lua_code_cache

Turn on or off the cache of Lua code in the *_by_lua_file (e.g. Set_by_lua_file,content_by_lua_file) instruction and in the Lua module.

If turned off, ngx_Lua creates a separate Lua VM for each request, all *_by_lua_file instruction code is not cached in memory, and all Lua modules are reloaded from scratch each time. In development mode, this gives us the convenience of debugging without needing a Reload nginx, but in a build environment, this is highly recommended. If turned off, even a simple Hello World will be an order of magnitude slower (expensive per IO read and compile).

However, code written directly to the *_by_lua_block directive in the nginx.conf configuration file is not updated in real time when you edit it, and can only be rewritten by sending an HUP signal to Nginx.

A small case

Dynamic routing is implemented through OpenResty + Redis

Nginx is often used as a reverse proxy server. Normally, we configure the back-end service in the upstream of Nginx. When the back-end service changes, we modify the configuration in the upstream and reload it to take effect. This can be a bit cumbersome if the back-end service changes frequently. The configuration from upstream is now placed in Redis using Lua + Redis to achieve dynamic configuration.

Architecture diagram

Principle:

Access_by_lua * replaces the proxy_pass address in the Nginx configuration by getting the corresponding backend service address from Redis using a specified rule (which is designed according to its own requirements).

Process:

1. Create the variable $backend_server for the back-end service address in the Nginx configuration.

server {listen 80; server_name app1.example.com; location / { ... set $backend_server ''; }}Copy the code

The address of the back-end service is also stored in Redis.

The set app1 10.10.10.10:8080Copy the code

1. Use ngx_redis2 module to realize an interface to read Redis.

# GET /get? key=some_keylocation = /get { internal; Set_unescape_uri $key $arg_key; # this requires ngx_set_misc redis2_query get $key; redis2_pass foo.com:6379; # redis_server and port }Copy the code

2. In the request access stage, the ngx.Location. capture module is used to request the Redis interface defined in the last stage, and the result is replaced with $backend_server.

location / { ... Server_ip local res = access_by_lua_block {local rds_key = "app1 ngx.location.capture('/get', {args = {key = rds_key}}) # parser = require("redis.parser") local server, typ = parser.parse_reply(res.body) if typ ~= parser.BULK_REPLY or not server then ngx.log(ngx.ERR, "bad redis response: ", res.body) ngx.exit(500) end ngx.var.backend_server = server } }Copy the code

3. The Nginx forwarding phase forwards the request to the back-end service.

location / { ... access_by_lua_block {... }; proxy_pass http://$backend_server; }Copy the code

Finally, two useful open source projects based on OpenResty are recommended:

• Gray scale publishing system ABTestingGateway based on dynamic strategy

• Web application firewall ngx_Lua_waf based on NGX_Lua

reference

• OpenResty Best Practices( https://legacy.gitbook.com/book/moonbingbing/openresty-best-practices/details )

• lua-nginx-module( https://www.nginx.com/resources/wiki/modules/lua/ )

• Nginx Lua Directives( https://github.com/openresty/lua-nginx-module#directives )

• Nginx API for Lua( https://github.com/openresty/lua-nginx-module#nginx-api-for-lua )

• Lua introductory tutorial (https://coolshell.cn/articles/10739.html)

The full text after