On August 31, 2019, the OpenRESTY community jointly held the OpenRESTY × Open Talk national tour salon · Chengdu station. Apisix author Wen Ming shared “API gateway selection and continuous integration” at the event.

OpenRESTY X Open Talk is a national touring salon initiated by the OpenRESTY China community, which invites senior OpenRESTY technical experts in the industry to share their OpenRESTY practical experience and promote communication and learning among OpenRESTY users. Drive the OpenResty open source project.

Wen Ming, co-founder of Shenzhen Tribranch Technology, Apisix PMC, founder of OpenRESTY Software Foundation, columnist of OpenRESTY From Beginning to Practice. Before starting his own business, he worked in Internet security companies for 10 years, mainly engaged in the development and architecture of server side. Responsible for the development of wooden Ma Ma killing, anti-phishing system and enterprise security products. He once served as an architect in Qihoo 360, the initiator and member of the open source committee.

Here’s the full text:

I am working on an open source API gateway project called Apisix with college students. Today I will introduce this project, which involves the technology and selection of OpenResty, mainly including three aspects:

  • First, what is an API gateway and what does it do;
  • Second, how to do the API gateway, how to do the selection;
  • Third, Apisix is an open source project that describes how we do testing and continuous integration with only two people. It covers some of the common things of OpenResty, which is worth learning from.

The role of the API gateway

What is an API gateway

Take Kong as an example. There is no API gateway in the picture on the left, but there are many services hanging behind it. If each service implements functions including authentication, statistics, security verification, etc., there will be a lot of repetitive work. API gateway function is extracted from these public things, such as the picture on the right, the following several services, each service only care about their business, and business has nothing to do all things into the API gateway, which is what the public API gateway such as statistics, security, current limiting, speed limit, and cache extracted made a middle tier.

Traditional functions of API gateways

  • Enables API requests to be handled more securely and efficiently. Traditional API gateways have some basic functions that are still applicable today. Managing both north-south and east-west API traffic can be handled quickly and safely, which was the original purpose of the API gateway.
  • Overrides all functionality of NGINX. The gateway to the API covers all Nginx functionality, including reverse proxying, load balancing, basic caching, secure authentication, flow limiting and speed limiting.
  • Support for Nginx and other Web servers can not achieve the function. Such as dynamic upstream, dynamic SSL certificate, dynamic current limit speed limit, as well as active/passive health check, service fuse, etc., these dynamic capabilities are traditional NGINX, Apache and other Web servers can not support.
  • Lifecycle management. In the API gateway space, the biggest player is Google, which in 2016 acquired a publicly traded company, Apigee, which now integrates the full functionality into the Google cloud. API gateway in addition to the familiar reverse proxy, load balancing, flow limit speed limit plug-in, it also includes the API design, documentation and testing, etc., which belong to the API gateway function, we called API life cycle management, from project design to test on-line, All of these things fall within the overall functionality of the API gateway.

New features under cloud native

Why are we doing it all over again now, including Kong and Apisix? This is because there are some new changes in the user and technology architecture under the cloud native and micro-service systems:

New changes in the technical architecture

  • It needs to connect with important components such as Prometheus, Zipkin and Skywalking in the cloud native.
  • GRPC Proxy and Protocol Conversion (REST<=> GRPC) : HTTP protocol is used less and less in microservices, many people are starting to use GRPC, that in the conversion from HTTP to GRPC protocol, this functionality is also needed, including GRPC proxy;
  • Identity authentication method change: in the traditional NGINX, the general traffic in according to the routing rules to do reverse proxy, load balancing, rarely on the identity of the client traffic authentication. But in the cloud native is different, because a lot of traffic is in the micro service, which needs to do strict identity authentication, including encryption, OpenID and other identity authentication. This is some new functions, you can put your own enterprise inside the need to do the identity authentication related things to the third party external certification manufacturers to do;
  • Serverless is also a concept that has become very popular in recent years. For example, you want to dynamically run a function or stop a function on the edge node, or dynamically change its contents. You can deploy your API gateway on the edge node, with this FaaS functionality. Your edge nodes will be more flexible. Apisix recently supported Serverless, which allows you to dynamically run a Lua function on an edge node;
  • Stateless, random scaling and scaling: API gateways were less demanding than they were a decade or so ago, when Internet traffic flowed more from the browser to the server, and there were no mobile phones, IoT devices, microservices, intranets, etc. However, the current traffic is very large, including 4G and 5G, under which there are many mobile phones and IoT devices to access the server, the traffic is particularly large. At this point, you need a higher performance API gateway to do support. One of the key standards of cloud native is that all services can be scaled and scaled at will by means of containers, friendly to Kubernetes;
  • Support for cloudy and mixed clouds: On cloud is now a trend, but we don’t usually will put the service only in a cloud, such as tencent, Google cloud cloud, ali cloud put part of the private cloud put part again, do the adjustment according to their security and price, some of the private higher on private cloud data security, resources such as which cheap put which related to the CDN cloud, Do the dynamic switch, then there will need to be a vendor-independent API gateway in front of the distribution.

application

Here are two examples of what the new API gateway does.

  • Replace all features of NGINX to handle north-south traffic

It has obvious advantages over Nginx. It is implemented completely dynamically. Whether it is Kong or Apisix, dynamic is the most basic point. Modifying any configuration file in NGINX requires Reload to take effect. Let’s imagine a scenario where we use NGINX to do the first routing, load balancing, sudden traffic comes, and we need to quickly add a lot of upstream servers, so we need to keep changing the NGINX configuration file, and then reload, and then remove the upstream service after the sudden traffic passes. Then change the Nginx file and reload it again. This cost is very high, but if you use Kong, Apisix, such a Web server, there will be no such problem, through the API can easily change the entire cluster of all machines upstream configuration, dynamic certificate configuration, of course, the performance will be slightly lower than Nginx. But this low is acceptable because it allows a lot of flexibility. Now many manufacturers are doing such a replacement, not only Internet companies, but also some traditional enterprises are slowly removing Nginx.

  • Zero Trust Gateway

One of the hottest concepts in the security world in the last two years is zero trust. In the traditional security field, we think that the boundary protection is very important, so we will use the firewall to do a layer of check on the incoming traffic, this check is actually a hit rule check, if it is black, it will be rejected. Then there is a problem: if some rules are not updated in time, it can pass through the firewall, before the security is more based on the border of protection, beyond the border of the internal network can be unimpeded.

But zero-trust gateways can completely solve this problem by assuming that all traffic is not secure and previously border protection was black and white. Today’s zero-trust security gateway is white or black, you’re either white, you’re black, so it’s completely identity based authentication. An API request will come to the authentication server, and the third-party authentication server such as Author0 or Okta will authenticate your identity. After the authentication, the request can be passed, or it will be rejected directly. This is a trend in the security field.

The needs of enterprise users

  • Not locked, can be retracted

The biggest demand of all enterprise users is: don’t lock them in. The API gateway is the gateway to the entire traffic, and users cannot be locked in it. For example, if you use the API gateway of Ali Cloud, in fact, you have been locked by it, because it is the entrance of traffic, you can not distribute to other cloud vendors. And then there’s the fallback, so I use this API gateway, like Nginx, so I can go back very easily, and that’s what we’re going to take into account when we’re doing Apisix. Users may want to return Nginx after using it for a while. We can also support this capability.

  • Keep the core stable

Although Apisix has a very fast iteration per month, with over 200 commits per month, and about 10 large features, there is a directory called core in Apisix that changes very little, and we will keep its core stable. Functional iterations can be fast, but the core is stable.

  • Support the development of enterprise personality requirements

Whether domestic or foreign enterprises, there is no way to take products to meet all the needs. If you accommodate all the requirements, then you end up with a giant, very large, very heterogeneous product, with very low performance and scalability. So we have plugins, and if you need to personalize something, you can customize your plugins, put it on if you need it, and take it off if you don’t need it. I can guarantee that the bottom core layer is stable, and the top layer can be customized to the user’s needs.

How to do gateway selection

Industry status

  • Refer to the Gartner report

This is our idea of selection. We do not choose technology first, but look at the whole industry first. The report of Gartner can be used as a reference. As can be seen in the chart above, leading players such as Google, IBM, and RedHat have taken over most of the global API market. Kong is known to the technical community, but he is actually a visionary. In addition, as an emerging player, Kong was the first to open source its API gateway.

  • Refer to the CNCF panorama

Gartner’s report is more commercial, and can be built on that by referring to the open source community’s Cloud Native Software Foundation (CNCF), which maintains a panoramic view of many open and closed source projects in a hierarchical, functional way. In the panorama, there are more than a dozen softwares in the cloud native field that can be selected for API gateway selection. Half of them are products of Dachang, such as 3Scale, Red Hat, and the rest are open source projects such as Kong, Apisix, Tyk, etc. If you want to make API selection, you can choose among them.

Comparison of product selection

After reviewing the Gartner report and the CNCF panorama, we compared the major API gateways and decided to build our own instead of an existing API gateway.

  • apigee

Apigee is the biggest player, and its advantage lies in the whole life cycle: from API design, development, documentation, testing, and launch, it’s all Google buckets. However, Apigee is a closed source project that cannot be customized and is locked by Google Cloud.

  • Kong

Kong solves the pain point of Apigee by neither locking nor supporting custom development. However, Kong was developed in 2015, when it put all the data in Postgres’ relatively heavy relational database, with complicated code and performance problems. Kong’s advantage lies in good product ideas and accurate direction.

  • APISIX

Apisix borrowed Kong’s idea and made the selection entirely based on ETCD, putting all the data in ETCD, and completing a lot of repetitive code that Kong had done on Postgres. For example, message distribution, high availability and extensibility were all based on ETCD, making the operation much simpler. The disadvantage of Apisix is that it has not been tested by large users since it was opened on June 6, 2019. However, it has obvious advantages: the secondary development is much easier than Kong. For example, the development of the function of flow limit and speed limit can be completed by adding 60 or 70 lines of code to the Apisix file, but Kong needs to modify 5 or 6 files and 200 or 300 lines of code.

How to do gateway technology selection?

Core components of the API Gateway:

  • Routes. Routes can be thought of as various locations in NGINX. It is a function of routing how to distribute locations to upstream services and how to load plug-ins. The choice is to do a traversal or a tree? The time required and complexity of the two are different, so we choose to traverse like Kong.
  • Plug-ins, which are the core functionality of the API Gateway, make it easy to develop your own code without waiting for the open source community to add functionality. When creating plug-ins, you need to think about whether or not to do hot loading. Hot loading means that when you add plug-ins or modify a particular plug-in, it can take effect without restarting the whole service. We want to make changes or add plug-ins that take effect immediately through API calls.
  • Schema check, Schema check refers to the user request to come in after the need to check whether the uploaded field, type and so on is legal, if you write a plug-in, its parameters, input values are legal, this is equivalent to the description of the API, make the individual and front end cooperation is very convenient.
  • Storage, whether it’s in a relational database or a key-value database or an ETCD is sort of a technology selection, and you have to pick these things up so that you can put them together like building blocks.

Selection principle: do cloud native friendly, high performance, open source API gateway

  • Be developer-friendly. The API gateway is used by developers, and we hope that it will be easier for developers to see the code, learn the code, or modify the plug-in than it was for me and the students to look at Kong’s code.
  • Go for performance. Although the performance of OpenResty is high, when you actually write the code, you will find that the performance of the code is not high, because OpenResty code is easy to write, but it is hard to guarantee that you will write the ultimate code, it has a lot of holes in it, which is very difficult for most people to notice.

The method of selecting APISIX

  • Routing: Lua-resty-r3 was selected at first, and FFI was added recently.
  • Plug-in: Inspired by Kong, but architecture and design are completely different, greatly simplifying the writing difficulty and hot loading;
  • Schema: Using Tencent’s open source RapidJSON, JSON Schema is implemented in RapidJSON, and JSON Schema standard is used for verification.
  • Storage: ETCD is selected. There is no Lua-resty-ETCD library at that time, so you need Lua to help ETCD access from scratch.

Apisix unique features

  • Super performance, performance is the main label, internal test found that the performance of Apisix is 10 times that of KONG;
  • Plugins hot update, modify or add plug-ins, do not restart the service, all the updates are hot;
  • Routes can be implemented as plug-ins. If you don’t like R3’s complex routes, you can use prefix-matching routes.
  • Support version change control, if the release of a new version error can easily back out of the old version;
  • With zero trust based on identity, the newly merged Future can support external authentication, where it is easy to implement zero trust and access the services of all external authentication vendors. Simply adjust some parameters when using, you can add the identity authentication function to API directly. The commercial version of Kong also supports this feature, but Apisix is directly open source.

Best practices for testing, continuous integration

test

Large companies have dedicated QA teams to do the testing, and the development department just writes the code, tests it, and submits it to the QA team. But open source projects don’t have a QA team, and even the development team is part-time, so you have to use automated testing, or test-driven development, to make open source projects work.

OpenResty has close to 70 open source projects, and its commercial companies have close to 100 closed source projects. In total, there are nearly 200 projects maintained by less than 10 people. If you test by hand, you can’t do anything, so find a way to implement this problem.

  • Development is Testing

It was slow at first, because after developing a new feature you had to submit the corresponding test cases, otherwise the PR would not be merged. For example, if you contribute a feature to OpenResty without submitting a corresponding test case, or if you submit an incomplete test case, no matter how well the feature is written, this PR will never be merged because it breaks the whole principle of open source projects that testing needs to be automated.

Apisix introduced an additional condition that code coverage should not be less than 70%, which has now been changed to no less than 80%. If your change introduces new code and also increases the test cases, but decreases the coverage of the original test cases, it will not be acceptable to merge.

  • The unit tests are entirely based on test:nginx

The test case is completely based on test:nginx. It can be considered that test:nginx is a small language or a DSL with fewer documents and a high learning threshold. Therefore, the test is completely based on the fact that the students and I are familiar with it.

  • Code style detection: Luackeck and Lua-Relang

Luackeck is very easy to use and can be used with Lua and OpenResty, with a few parameters to choose from and Lua-Relang written using brother chun. We run both programs to do code style checks to ensure that both new submitters and our own code styles are consistent.

  • Code coverage detection: Luacov

Code coverage detection uses Luacov, a feature of standard Lua that allows code coverage to run.

Merge PR is carried out on the assumption that the above tests (test:nginx test, code style test, code coverage test) have been run, and the performance test is carried out on the 6th of every month when the new version is released to compare the performance difference between the old version and the new version. General will run the flame chart, at the same time do regular fuzzing test, chaos input such as URI, args and so on to do pressure test. Apisix is, after all, an API gateway, and as an entry point for traffic, it needs to be stable enough.

Continuous integration

The test cannot rely on people, or one day it will not run. So how do you make unit tests, performance tests, code style checks, and so on run steadily without relying on people?

  • Strong reliance on GitHub: Issue, Milestone, Code Review, PR Approves. GitHub’s Code Review is very handy because it can comment on each line of code, and there is no way to merge Approved if it thinks it needs to be changed and it hasn’t.
  • Strong reliance on Travis CI: unit testing, code style detection, multi-platform testing (Ubuntu and Mac), front-end packaging, auto-commit. Travis is a continuous integration plugin for GitHub. Apisix does not use its own server resources for testing, but runs in the cloud. It now has a front-end for Apisix, and its packaging and submission are automated. Since we are compiling from Vue code to HTML, we are doing all this through Travis.
  • http://coveralls.io, and upload the results of code coverage to the “tie” website to visualize everything related to code coverage, such as which line of code was not detected, and what changes were made in the previous version, so that code coverage detection can be automated. As mentioned earlier, to keep Apisix core stable at all times, you need to make sure that all files in the core directory have 100% code coverage, meaning that every line of code in the core is tested.

conclusion

  • Less resources is not necessarily a bad thing. Whether open source projects or commercial companies, only when resources are scarce will they think about how to use external forces or come up with some clever ideas to solve problems.
  • Apisix’s selection, testing, and CI are all “clever” and automated, with the help of clever external forces, without making wheels;
  • Apisix’s selection, testing and CI are very important, more important than performance. Although we focus on performance, it’s worth thinking about not only the code, but also the selection of how to test and CI. You can apply it to your business, even without OpenResty and Apisix, you can still learn how to do open source projects.
  • GitHub and SaaS offer services that you would never build yourself. Resist the urge to build your own wheels.

View the PPT and video of the speech:

API gateway selection testing and continuous integration

This article by the blog multiple platform
OpenWriteRelease!