The previous article introduced Kong’s practices and links. This article will introduce Kong’s best tools: plug-ins and custom plug-ins.
Kong several common plug-in applications
When the request arrives at Kong, we can apply Kong’s own plug-in to process the request before forwarding it to the server application, such as legitimate authentication, traffic limiting control, blacklist and whitelist verification, log collection, etc. At the same time, we can also follow Kong’s tutorial documentation and customize our own plug-ins. This section will select two of these plug-in sample applications. For the remaining plug-in applications, see docs.konghq.com/hub/.
JWT certification plug-in
JWT is currently the most popular cross-domain authentication solution. As an open standard (RFC 7519), it defines a concise, self-contained method for securely transferring information between communicating parties in the form of JSON objects. The information is trusted because of the digital signature.
Why JWT is used will not be discussed in detail in this section, but the design and practice of unified authentication and authorization in microservices architecture can be seen in detail. Kong provides a JWT authentication plug-in to validate requests from JWT that contain HS256 or RS256 signatures (as described in RFC 7519). Each consumer will have JWT credentials (public and key) that must be used to sign their JWT. JWT tokens can be passed through request strings, cookies, or authentication headers. Kong will verify the token’s signature and forward it if it passes, otherwise the request will be discarded.
We add the JWT authentication plug-in to the routing configured in the previous section.
curl -X POST http://localhost:8001/routes/e33d6aeb-4f35-4219-86c2-a41e879eda36/plugins \
--data "name=jwt"
Copy the codeAs you can see, the corresponding record has been added to the plug-in list.
With the addition of the JWT plug-in, you cannot access the/API /blog interface directly. The interface returns: “message”: “Unauthorized”. Prompting the client to provide JWT authentication information to access. Therefore, we need to create users:
curl -i -X POST \
--url http://localhost:8001/consumers/ \
--data "username=aoho"
Copy the codeA user named AOho has been created.
curl -i -X POST \
--url http://localhost:8001/consumers/aoho/jwt \
--header "Content-Type: application/x-www-form-urlencoded"{/ / response"rsa_public_key": null,
"created_at": 1563566125,
"consumer": {
"id": "8c0e1ab4-8411-42fc-ab80-5eccf472d2fd"
},
"id": "1d69281d-5083-4db0-b42f-37b74e6d20ad"."algorithm": "HS256"."secret": "olsIeVjfVSF4RuQuylTMX4x53NDAOQyO"."key": "TOjHFM4m1qQuPPReb8BTWAYCdM38xi3C"
}
Copy the codeJWT credential information can be generated at https://jwt.io using key and secret. In the actual use process, we achieve through coding, here to demonstrate the use of web tools to generate tokens.
Configure the generated Token to the authentication header of the request and execute the request again:
As you can see, we can request the corresponding API interface normally. The JWT authentication plug-in is successfully applied.
Prometheus visual monitoring
Prometheus is an open source system monitoring and alarm framework. It was inspired by Google’s Borgmon monitoring system, created in 2012 by former Google employees working at SoundCloud, developed as a community open source project and released in 2015. In 2016, Prometheus joined the Cloud Native Computing Foundation as the second most popular project after Kubernetes. Prometheus is a new generation monitoring framework for logging time series data, featuring a powerful multidimensional data model, flexible and powerful query statements, and ease of management and scaling.
The metrics for Prometheus are as follows:
- Status code: HTTP status code returned by the upstream service.
- Delay bar: All delays in Kong will be recorded, including the following:
- Request: delay of complete request;
- Kong: The time taken by Kong to route, validate, and run other plug-ins;
- Upstream: The time taken by the upstream service to respond to a request.
- Bandwidth: total Bandwidth (egress/inlet) flowing through Kong;
- DB reachability: Whether Kong node can access its DB;
- Connections: Various NGINX connection metrics, such as Active, read, write, and accept Connections.
We installed Prometheus plugin on aoho-blog Service:
curl -X POST http://localhost:8001/services/aoho-blog/plugins \
--data "name=prometheus"
Copy the codeAs you can see from the admin interface, we have successfully bound the Prometheus plug-in to the AOho-Blog service.
Return collected metrics by accessing the /metrics interface:
HTTP / 1.1 200 OK $curl -i http://localhost:8001/metrics Server: openresty / 1.13.6.2 Date: Sun, 21 Jul 2019 09:48:42 GMT Content-Type: text/plain; charset=UTF-8 Transfer-Encoding: chunked Connection: keep-alive Access-Control-Allow-Origin: * kong_bandwidth{type="egress",service="aoho-blog"} 178718
kong_bandwidth{type="ingress",service="aoho-blog"} 1799
kong_datastore_reachable 1
kong_http_status{code="200",service="aoho-blog"} 4
kong_http_status{code="401",service="aoho-blog"} 1
kong_latency_bucket{type="kong",service="aoho-blog",le="00005.0"} 1
kong_latency_bucket{type="kong",service="aoho-blog",le="00007.0"} 1... kong_latency_bucket{type="upstream",service="aoho-blog",le="00300.0"} 4
kong_latency_bucket{type="upstream",service="aoho-blog",le="00400.0"} 4... kong_latency_count{type="kong",service="aoho-blog"} 5
kong_latency_count{type="request",service="aoho-blog"} 5
kong_latency_count{type="upstream",service="aoho-blog"} 4
kong_latency_sum{type="kong",service="aoho-blog"} 409
kong_latency_sum{type="request",service="aoho-blog"} 1497
kong_latency_sum{type="upstream",service="aoho-blog"} 1047
kong_nginx_http_current_connections{state="accepted"} 2691
kong_nginx_http_current_connections{state="active"} 2
kong_nginx_http_current_connections{state="handled"} 2691
kong_nginx_http_current_connections{state="reading"} 0
kong_nginx_http_current_connections{state="total"} 2637
kong_nginx_http_current_connections{state="waiting"} 1
kong_nginx_http_current_connections{state="writing"} 1
kong_nginx_metric_errors_total 0
Copy the codeThe metric provided by the Prometheus plug-in is reflected in the response which is too long and omitted. Metrics derived from the Prometheus plug-in can be plotted in Grafana for readers to try.
Link-tracking Zipkin plug-in
Zipkin is an open source distributed real-time data tracking system. Its main function is to gather real-time monitoring data from various heterogeneous systems to track system delay in microservices architecture. Applications need to report data to Zipkin. Kong’s Zipkin plug-in, as a Zipkin-client, is to assemble the data packets needed by Zipkin and send data to Zipkin-server. The Zipkin plugin labels the request as follows and pushes it to the Zipkin server:
- Span. Kind (sent to Zipkin as “kind”)
- http.method
- http.status_code
- http.url
- peer.ipv4
- peer.ipv6
- peer.port
- peer.hostname
- peer.service
For more information on link tracing and Zipkin, see Full link Tracing in microservices Architecture. This chat aims to show you how to track all requested links using the Zipkin plug-in in Kong.
Start by opening the Zipkin plug-in and bind the plug-in to the route (in this case, it can be a global plug-in).
curl -X POST http://kong:8001/routes/e33d6aeb-4f35-4219-86c2-a41e879eda36/plugins \
--data "name=zipkin" \
--data "config.http_endpoint=http://localhost:9411/api/v2/spans" \
--data "config.sample_ratio=1"
Copy the codeThe Zipkin Collector address and sampling rate have been configured above. In order to be effective, set the sampling rate to 100% and use it with caution in production environment. The sampling rate has an impact on system throughput.
As you can see, the Zipkin plug-in has been applied to the specified route. Now we will implement the request/API /blog interface and open the http://localhost:9411 interface as follows:
Zipkin has logged the request, so we can click on it to see the link details:
The link call tells you which services and spans the request went through after it arrived at Kong, how long each Span took, and so on.
The practice of customizing plug-ins
There are many plug-ins available, but we still have business requirements in real business scenarios, and custom plug-ins can help us better manage the API Gateway. Kong provides the plug-in development kit and examples, so you can simply follow the provided steps to customize your plug-in.
Kong installation
In the above section, the author introduces how to install Kong by mirroring. In this part, to facilitate the compilation of custom plug-ins, we use the locally installed Kong. The author’s environment is macOS, and the installation is relatively simple:
$ brew tap kong/kong
$ brew install kong
Copy the codeSecond to install Postgres, and download kong. Conf. The default configuration file (see raw.githubusercontent.com/Kong/kong/m…
$ sudo mkdir -p /etc/kong
$ sudo cp kong.conf.default /etc/kong/kong.conf
Copy the codePerform migration:
kong migrations bootstrap -c /etc/kong/kong.conf
Copy the codeThen you can start Kong:
kong start -c /etc/kong/kong.conf
Copy the codeAfter the startup, verify whether the management port 8001 is successful.
curl -i http://localhost:8001/
Copy the codeBased on the installed Kong, we introduce how to add custom plug-ins to optional plug-ins of Kong. Here, we take the token-Auth plug-in for authentication as an example.
Kong officially provides plug-ins related to authentication, such as JWT, OAuth 2.0 and Basic Auth, etc. In our actual business, we often build our own authentication and authorization server, which requires us to intercept the validity of the authentication request at the API gateway. Based on this, we implement a plugin similar to the Kong filter: token-Auth.
Kong’s own plug-in in/usr/local/share/lua / 5.1 / Kong/plugins/directory. There are two main files in each plug-in folder:
- Schema. lua: defines parameter checks for plug-in startup;
- Handler. lua: The file defines the functions to be executed at each stage, the heart of the plug-in.
Token-auth is our custom plugin name. In the/usr/local/share/lua / 5.1 / kong new token – auth directory under the/plugins. Module. lua and handler.lua are loaded in the Plugin directory during the loading and initialization phase, i.e. kong.init ().
Plug-in configuration definition: schema.lua
The configuration of each plug-in in Kong is stored in the plugins table config field, which is a JSON text. The configuration required by token-Auth is defined as follows:
return {
no_consumer = true,
fields = {
auth_server_url = {type = "url", required = true}}},Copy the codeAs you can see from schema. Lua, when the token-Auth plug-in is enabled, you need to check that the AUTH_server_URL field is of URL type and cannot be empty.
Plugin function implementation: handler.lua
Handler. lua implements plug-in authentication. Methods defined in this plug-in are called when processing requests and responses.
llocal http = require "socket.http"
local ltn12 = require "ltn12"
local cjson = require "cjson.safe"
local BasePlugin = require "kong.plugins.base_plugin"
local TokenAuthHandler = BasePlugin:extend()
TokenAuthHandler.PRIORITY = 1000
local KEY_PREFIX = "auth_token"
local EXPIRES_ERR = "token expires"-- Extract JWT header information -- @param Request NGX Request object -- @return token JWT
-- @return err
local function extract_token(request)
local auth_header = request.get_headers()["authorization"]
if auth_header then
local iterator, ierr = ngx.re.gmatch(auth_header, "\\s*[Bb]earer\\s+(.+)")
if not iterator then
return nil, ierr
end
local m, err = iterator()
if err then
return nil, err
end
if m and #m > 0 then
returnM [1] end end end -- Call auth Server to verify the validity of token -- @param token To be validated -- @param conf Plugin configuration -- @return info Information associated with token
-- @return err
local function query_and_validate_token(token, conf)
ngx.log(ngx.DEBUG, "get token info from: ", conf.auth_server_url)
local response_body = {}
local res, code, response_headers = http.request{
url = conf.auth_server_url,
method = "GET",
headers = {
["Authorization"] = "bearer ". token }, sink = ltn12.sink.table(response_body), }if type(response_body) ~= "table" then
return nil, "Unexpected response"
end
local resp = table.concat(response_body)
ngx.log(ngx.DEBUG, "response body: ", resp)
if code ~= 200 then
return nil, resp
end
local decoded, err = cjson.decode(resp)
if err then
ngx.log(ngx.ERR, "failed to decode response body: ", err)
return nil, err
end
if not decoded.expires_in then
return nil, decoded.error or resp
end
if decoded.expires_in <= 0 then
return nil, EXPIRES_ERR
end
decoded.expires_at = decoded.expires_in + os.time()
return decoded
end
function TokenAuthHandler:new()
TokenAuthHandler.super.new(self, "token-auth"End -- Implement the Access methodfunction TokenAuthHandler:access(conf)
TokenAuthHandler.super.access(self)
local token, err = extract_token(ngx.req)
if err then
ngx.log(ngx.ERR, "failed to extract token: ", err)
return kong.response.exit(500, { message = err })
end
ngx.log(ngx.DEBUG, "extracted token: ", token)
local ttype = type(token)
if ttype ~= "string" then
if ttype == "nil" then
return kong.response.exit(401, { message = "Missing token"})
end
if ttype == "table" then
return kong.response.exit(401, { message = "Multiple tokens"})
end
return kong.response.exit(401, { message = "Unrecognized token" })
end
local info
info, err = query_and_validate_token(token, conf)
if err then
ngx.log(ngx.ERR, "failed to validate token: ", err)
if EXPIRES_ERR == err then
return kong.response.exit(401, { message = EXPIRES_ERR })
end
return kong.response.exit(500,{ message = EXPIRES_ERR })
end
if info.expires_at < os.time() then
return kong.response.exit(401, { message = EXPIRES_ERR })
end
ngx.log(ngx.DEBUG, "token will expire in ", info.expires_at - os.time(), " seconds")
end
return TokenAuthHandler
Copy the codeThe token-Auth plug-in implements two methods, new() and Access (), and only works in the Access phase. In the Access () method, the JWT header information is first extracted to check whether the token exists and is in the correct format, etc., and then the authentication server is requested to validate the token.
Load the plug-in
After the plug-in is developed, create the token-auth-1.2.1-0. Rockspec file in the plug-in directory and fill in the newly developed plug-in:
package = "token-auth"
version = "1.2.1-0"
supported_platforms = {"linux"."macosx"}
local pluginName = "token-auth"
build = {
type = "builtin",
modules = {
["kong.plugins.token-auth.handler"] = "kong/plugins/token-auth/handler.lua"["kong.plugins.token-auth.schema"] = "kong/plugins/token-auth/schema.lua",}}Copy the codeThen add the newly developed plug-in to the kong.conf configuration file:
$ vim /etc/kong/kong.conf
# Remove the leading comment and modify it as follows
plugins = bundled, token-auth
Copy the codeThe Bundled attribute means the official collection of plug-ins, enabled by default. Here, we added a custom token-Auth plug-in. Verify that the custom plug-in loaded successfully:
${curl http://127.0.0.1:8001/plugins/enabled"enabled_plugins": ["correlation-id"."pre-function"."cors"."token-auth"."ldap-auth"."loggly"."hmac-auth"."zipkin"."request-size-limiting"."azure-functions"."request-transformer"."oauth2"."response-transformer"."ip-restriction"."statsd"."jwt"."proxy-cache"."basic-auth"."key-auth"."http-log"."datadog"."tcp-log"."post-function"."prometheus"."acl"."kubernetes-sidecar-injector"."syslog"."file-log"."udp-log"."response-ratelimiting"."aws-lambda"."bot-detection"."rate-limiting"."request-termination"]} %Copy the codeTo enable the plugin
To enable the token-auth plugin on the Service, specify the config. auth_server_URL property:
$ curl -i -XPOST localhost:8001/services/aoho-blog/plugins \
--data 'name=token-auth' \
--data 'config.auth_server_url=<URL of verification API>'
Copy the codeIf your plug-in has its own database tables or has requirements on database tables or data in them, create the Migrations directory in the plug-in directory. Create migrations/postgres.lua or migrations/cassandra.lua, depending on whether you are using Postgres or Cassandra.
If the plug-in has its own database tables, you also need to create daos.lua in the plug-in directory, returning the database table definition. If there is no separate database table, you do not need to create this file.
Here is not to do too much demonstration, readers can combine the author’s previous chat: unified authentication and authorization in the micro-service architecture design and practice, build authentication and authorization server, have a try.
summary
Gateway is an indispensable basic service in microservices architecture. This article describes how to use Kong to build microservices gateway. Kong is a modern cloud native gateway with excellent ease of use and performance compared to other gateway components. Some of Kong’s plug-in usage was then introduced. Kong official and community provides a rich API gateway plug-in, configuration can be used. Finally, the author implements a custom token-Auth plug-in in this paper. Kong’s open plug-in mechanism enables developers to flexibly implement special business requirements.
Recommended reading
API Gateway practices in cloud native architecture
Subscribe to the latest articles, welcome to follow my official account
