Opportunities and challenges born from cloud Yuan
Standards and ecological implications
Since The release of Kubernetes V1.0 on July 21, 2015 and the establishment of the CNCF organization, both Kubernetes and CNCF have experienced rapid development in the following years. Today, Kubernetes has entered the mature stage, and the cloud native era has just begun. Although cloud native is not just the ecology around Kubernetes, there is no doubt that Kubernetes is already the cornerstone of cloud native ecology. Through specification apis and CRD standards, Kubernetes has built a cloud native PaaS ecosystem that has become the de facto standard in the PaaS space.
This layer of de facto standards has enormous implications for enterprise delivery. Before the emergence of K8s ecology, similar to civil engineering, even screw nuts and other things are lack of unified standards, and if the enterprise party A only focuses on the upper business functions, it is easy to put the high platform on the floating sand, leading to the collapse of business. It is no exaggeration to say that in the field of enterprise delivery, K8s are like night.
Take the API routing function in API management as an example. If K8s is not used, enterprises may choose various gateway software such as F5, Nginx, HAProxy, and Zuul to configure the corresponding routes. Some software provides console UI, some may be human script operation and maintenance, lack of standards, operation and maintenance skills can not precipitate, the departure of key personnel may bring disaster. K8s abstracts the capability of API routing into Ingress resources, defines standards and shields the details of the underlying software. Personnel certified by CNCF CKA will have the capability of API routing operation and maintenance. In the field of API management, in addition to API routing, there are also API traffic governance policy, API open authentication, and call volume observation audit, etc. K8s and Istio ecology have provided some standard definitions, although many of them are not mature, but the future of standards and ecology is becoming increasingly clear. Erda Cloud has been firmly on this path, providing enterprises with standards-compliant, trusted API management products.
Different cloud native gateways argue
Gateways are a key component of an API management product. In the cloud native ecosystem, there are many different Ingress providers for THE K8s Ingress standard. Someone put together this chart:
As you can see from the table, the underlying gateway implementations of these Ingress providers are: Nginx, HAProxy, Envoy, Traefik, and Skipper.
As the official implementation, Kubernetes Nginx Ingress has the largest ecosystem of users and is currently the most used in mass production deployments. Before the emergence of cloud native ecology, Nginx has been sitting on the first position of the gateway in the Web 2.0 era with high performance and easy expansion, and has a good development and operation ecology, so it is selected by K8s as the default Ingress Provider, and then with Ingress, The ecology will be further expanded.
HAProxy and Nginx have similar experiences. They both emerged in the era of Web 2.0 and occupied a place in the gateway ecosystem with high performance and very low resource consumption. Before the combination with K8s Ingress, HAProxy and Nginx play a more similar role as ADC (Application Delivery Controller) access layer load balancer in software system architecture. Even if the microservice architecture is adopted, Nor is it directly responsible for microservice traffic exposure, usually by adding an additional layer of microservice Gateway forwarding, such as the Spring Cloud Gateway. Because microservice gateways often need to communicate with registries in microservice architectures for service discovery, which is not the strong suit of HAProxy and Nginx, there is no better choice in Spring Cloud microservices than Spring Cloud Gateway. However, K8s changes all that by providing a Service discovery mechanism that is compatible with Ingress/Service/Endpoint, and relying on the health check mechanism built in from code, it can achieve more universal Service discovery without language restrictions and has better scalability. This has helped HAProxy and Nginx expand their tentacles into microservice governance.
Even before K8s, there were gateway software attempts to expand from traditional load balancing to microservices governance, and Kong was the leader. With the help of OpenResty, on the basis of Nginx’s high performance and reliability, it can realize the rapid development of business function plug-ins. So Kong has implemented its own service discovery and health check mechanism.
However, it is a pity that after the emergence of K8s, the independent health inspection mechanism became useless. Kong Ingress still retains this useless mechanism, which is not a wise choice. In addition, Kong Ingress only supports its own Lua plugin ecosystem in routing policy configuration, and does not support native Nginx configuration instructions. Although Kong’s plugin ecosystem is very perfect, it can handle things like adding HTTP reply headers with one line of Nginx configuration instructions. Kong’s plugin is a bit of a low-level duplication, jumbled up and inefficient. However, Kong has a rich plugin ecosystem, especially mature authentication and authentication functions, which is the main reason why Erda Cloud uses Kong for some API management capabilities even though it didn’t choose Kong Ingress.
A software life cycle, if aligned with the architectural ecological development cycle, will burst out dazzling light. If Kong did not meet K8s at the best age, “I was born before you were born, you were born before I was old”. Envoy and Istio complement each other, making a perfect couple. Istio expands the capabilities of Envoy gateway from north-south management of entries to east-west traffic management between microservices, and the configuration dynamic update mechanism based on xDS protocol is superior to HAProxy and Nginx. However, Envoy configuration files are described in JSON and are not self-descriptive, so Istio/Ambassador/Gloo and others are secondary abstractions of configuration to facilitate operation and maintenance. While Istio + Envoy has the largest future imagination, Gloo is currently doing the best in terms of coverage of gateway capabilities based on configuration abstractions, but still lags behind Nginx Ingress in terms of adC-like capabilities. The coverage of gateway capabilities is closely related to the promotion of ecological needs, because the degree of abstraction of the configuration is not high enough for Ingress Providers to perform secondary abstraction to the envoys themselves, and to some extent, they separate the ecology to catch up with the ecology of Nginx Ingress. There’s still a long way to go.
Golang-based upstarts such as Traefik and Skipper are not to be ignored. While the user ecosystem is not as strong as the older ones, it has a strong developer ecosystem, thanks to Golang’s language that makes concurrency and IO so easy. There are two sides to pandering to developers. For example, with GC, it becomes more difficult to maintain an ideal memory footprint in the face of a large number of concurrent connections and requests, and to achieve the performance of C/C++ gateways increases the complexity of code design to the extent that it is no longer easy to rely on.
The reason for the contention among cloud native gateways is that none of the products can fully cover the needs of users, from adC-like to microservice discovery to authentication and authentication, and other more business-close API management domains. Erda Cloud’s Cloud native API management solution also uses Nginx Ingress + Kong’s two-tier architecture. However, I believe that with the development of standards and the expansion of gateway boundaries, there will be a single-layer full-stack gateway software that can not only fully meet the requirements of the underlying network operation and maintenance, but also can well expand the support of the upper business strategy, and it must be oriented to the cloud native architecture.
API management products have their cake and eat it
Around 2010, Apigee and MuleSoft’s API management products appeared and gave the concept of API lifecycle management, aiming to get through the whole process from design and development, test deployment, open integration, and finally realize API monetization. Today, consulting firms such as Gartner also regard API lifecycle management as a key part of enterprise digitization and produce annual analysis reports to provide consulting services to enterprises. Below is the Magic Quadrant of Gartner API Lifecycle Management for 2020:
As the chart above shows, Apigee and MuleSoft are firmly in the leadership quadrant thanks to their first-mover advantage. You can see That Kong is also in the leadership quadrant, even ahead of Apigee and MuleSoft at a strategic level, largely due to its rich business plug-in ecosystem and cloud-native designs like Kong Ingress that are ahead of established API management offerings. However, compared to mature API-managed SaaS products such as MuleSoft, Kong has not been a breakout product.
Existing mature API management SaaS products all have their own API gateways implemented as kernels. However, different from the cloud native Ingress gateway, these API gateways are not directly routed to THE K8s service, but are built on the whole business system to do API management at the application system level. This also means that once you choose such SaaS products, you have to accept the overhead of an extra layer of forwarding over the network, and possibly across the public network, the overhead of TLS encryption transmission, which can add tens to hundreds of milliseconds of latency to the network transmission.
The full life cycle API management product provided by Erda Cloud not only takes advantage of Kong’s mature API management plug-in ecosystem, but also combines Nginx Ingress to connect directly to Cloud native applications faster and more conveniently. Users don’t have to choose between latency overhead and API management capabilities. This is the biggest difference between Erda Cloud’s cloud-native API management product and others on the market.
Erda Cloud Cloud native gateway
Introduction of architecture
Considering function coverage, user ecology and enterprise production practice scale, the gateway product of Erda Cloud chooses Nginx Ingress as the K8s Ingress Provider. Erda Cloud can host K8s clusters built by users (or using Cloud vendor K8s services), as long as the Nginx Ingress Controller version installed in the cluster is higher than 0.25, you can directly use the Erda Cloud native gateway product.
Erda Cloud uses Kong as an extension that users can install when they have business level API management requirements such as API open authentication. When users configure the corresponding domain name /API function requirements on Erda Cloud, Kong will automatically take over the corresponding Ingress and re-route to the corresponding K8s Service. Based on Unix Domain socket/eBPF sockmap, the delay of Nginx Ingress forwarding to Kong can be reduced to negligible.
Of course, adding a layer of proxy forwarding will definitely bring additional resource occupation, which is the deficiency of this architecture. However, with the advantages of the cloud-native Ingress Provider for users’ smooth access, the ability to cover everything from adC-like functions to business layer API management functions, and the flexible design that users can independently manage which traffic is forwarded through Kong, The trade off is still worth doing.
Highlight the length of Ingress Annotation
Nginx Ingress can implement powerful traffic governance features based on Ingress annotations, such as the ability to limit the request rate:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-example-host annotations: # each IP can access nginx 5 times per second. The ingress. Kubernetes. IO/limit - RPS: 5 # as the calculating speed of bucket algorithm into the size, and limit the multiplication coefficient of RPS nginx. Ingress. Kubernetes. IO/limit - into - multiplier: 5 # with limit - window said global speed limit of 100 times per second (need to rely on memcached) nginx. Ingress. Kubernetes. IO/global - rate - limit: 100 nginx.ingress.kubernetes.io/global-rate-limit-window: # 1 s limit sent to the back-end service delivery rate of 1 MB per second nginx. Ingress. Kubernetes. IO/limit - rate: 1024 # 10 MB data before sent to the back-end service without limit nginx. Ingress. Kubernetes. IO/limit - rate - after: 10240 spec: rules: - host: "foo.bar.com" http: paths: - path: "/bar" backend: service: name: bar port: number: 80Copy the code
Request speed limit is the tip of the iceberg of Ingress Annotation capabilities. This simple configuration is unmatched by many other gateways, such as Istio Ingress Gateway, which has to be implemented through the complex EnvoyFillter configuration. Gloo makes some configuration simplifies. but it also puts some of this functionality in the commercial version, which is not available in the open source version.
Erda Cloud further enhances the ability of Ingress Annotation. To limit the request rate, for example, Nginx Ingress relies on external memcached for synchronization between multiple instances of Nginx Ingress. Erda Cloud can realize the awareness of the number of Nginx Ingress instances and automatically update the speed limit configuration. For example, if the gateway policy of the Erda Cloud is configured with a limit of 100 requests per second, and the current number of Nginx Ingress instances is 2, the actual speed limit configured in the Ingress is 50 requests per second. More than that, Erda Cloud reduces the mental cost of doing such a configuration for users. If you are familiar with Nginx, you should know that the request rate limit is based on the leaky bucket algorithm. Burst size is the key to achieve peak filling. If the burst size is 0, the occasional request interval will also be denied. But ordinary users might not understand this mechanism, the Nginx Ingress Nginx. Ingress. Kubernetes. IO/limit – into – multiplier configuration items will be.
Let’s take a look at how Erda Cloud lets users do configuration.
The gateway configuration of the Erda Cloud replaces the burst configuration with the maximum additional latency based on a simple formula:
Burst Size = Max Extra delay (s) * Max Throughput (requests/s)Copy the code
The so-called maximum extra delay is much easier to explain: when the request rate exceeds the specified rate, the peak filling valley is carried out, and additional delay is added for the current request. If the additional delay required exceeds the specified maximum extra delay, the access is directly denied.
Nginx Ingress does not provide out-of-the-box annotations to support this local mode global rate limit, but it can still customize Nginx configurations based on snippet annotations:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-example-host annotations: # will add the following under the corresponding server {} nginx configuration snippet nginx. Ingress. Kubernetes. IO/server - the snippet: | ###ERDA-AUTO-CONFIG### location @server-guard-65feeb { more_set_headers 'Content-Type: text/plain; charset=utf-8'; return 429 "System is busy, please try it later."; } # # # # # # ERDA - AUTO - CONFIG# will be in the corresponding location under add {} to nginx configuration fragments nginx. Ingress. Kubernetes. IO/configuration - snippet: | ###ERDA-AUTO-CONFIG### limit_req zone=server-guard-65feeb burst=5; limit_req_status 581; error_page 581 = @server-guard-65feeb; ###ERDA-AUTO-CONFIG### spec: rules: - host: "foo.bar.com" http: paths: - path: "/bar" backend: service: name: bar port: number: 80Copy the code
When a request is rejected, an internal 581 status code is returned, and an error_page is used to jump to a user-defined location. In this named location, the rejection status code and the rejection body are returned.
Because the limit_req_zone directive can only be configured within the HTTP block of Nginx, Erda Cloud updates the corresponding Ingress Annotation, Configmap nginx-Configuration also needs to be updated:
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configuration
namespace: kube-system
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
data:
http-snippet: |
###ERDA-AUTO-CONFIG###
limit_req_zone 1 zone=server-guard-65feeb:1m rate=100r/s;
###ERDA-AUTO-CONFIG###
Copy the code
These, Erda Cloud generated configurations are wrapped in ### erda-auto-config-config ### comment blocks, outside of which users can still add custom configuration items that are not affected by the Erda Cloud configuration.
From this example, you can see the power of Nginx and Ingress together. With snippet snippets, you can use Nginx’s native configuration to the fullest in Ingress. This is also the beauty of cloud native, which takes Nginx configuration operations to a whole new level, better decoupled and better suited to microservices architectures. Erda Cloud goes one step further and reduces the mental cost of these configurations to r&d operations personnel through product packaging.
Take the strength of Kong to complement the weakness of Ingress
Although Nginx Ingress relies on K8s Service to realize microservice discovery and routing, it lacks the ability to manage microservice API because it cannot support HTTP Method level routing control. For example, if a service only wants to expose a specific API GET/API /assets/{assetID}, the routing mechanism based on Nginx Ingress cannot be implemented. This requirement can be easily implemented on the Cloud native gateway of the Erda Cloud.
First, add the API to be exposed to the corresponding service in the microservice API management. The following figure shows only one API exposed: GET/API /assets/{assetID}.
Then specify the domain URL routing to the micro service, such as domain name for the helloworld. Erda. Cloud, according to the following configuration, when the request the helloworld. Erda. Cloud/API/example/assets / 100, The gateway rewrites the request path to/API/Assets /100 and forwards it to the back-end microservice.
This function is achieved with the help of Kong. When the user creates the microservice API, it associates the microservice with Kong’s Route/Service. When a user configates a route on a specific domain name, an Ingress is created to forward to Kong:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /c517f0f2/api/assets$1
nginx.ingress.kubernetes.io/upstream-vhost: dev.gateway.inner
nginx.ingress.kubernetes.io/use-regex: "true"
name: erda-route-ed71b6-6de9f1
namespace: addon-kong-pb8210
spec:
rules:
- host: helloworld.erda.cloud
http:
paths:
- backend:
service:
name: kong
port:
number: 8000
path: /api/example/assets(.*)
pathType: ImplementationSpecific
Copy the code
With the help of Kong, Nginx Ingress also makes up for its lack of authentication ability. Kong provides rich AuthN plug-ins. Erda Cloud mainly uses OAuth2, Key Auth and HMAC Auth plug-ins. It is possible to manage callers’ credentials for different AuthN plug-ins uniformly in the Erda Cloud:
You can authorize the entire domain name or specific routes under the domain name to different callers:
Open authentication of API is a key capability of enterprise API management. Erda Cloud hopes to better meet this requirement and make it more convenient for providers to open API. Let the caller can apply for the call, automatically obtain credentials and SLA quota limits; Thus, the overall efficiency of API management is improved. Erda Cloud builds on this Cloud native gateway capability to launch products for API lifecycle management.
Erda Cloud API management products
The Erda Cloud API management product consists of three modules: API design Center, API marketplace, and API access management.
The interface provider designs the interface in the API design center and then publishes it to the API marketplace. The interface caller looks for the interface in the API bazaar and applies for the calling credentials and SLA quota. The interface provider, or manager, authorizes callers’ access in API access management and can audit and analyze request traffic from individual callers.
API Design Center
Some API design products on the market will require users to fully understand swagger 2.0/ OpenAPI 3.0 protocol, to type out a document word for word; Other products support API design in a more friendly way, but do not use standard protocols, and their output documents lose versatility.
The Erda Cloud API Design Center is based on visual editing, with an intuitive and user-friendly interface that allows users to easily write professional API documents without any knowledge of REST API specifications or API description language. At the same time, it adopts OpenAPI 3.0 protocol standard, which can deliver and move out documents at any time, design once and use anywhere. API documents hosted on other platforms, Swagger files generated in code, etc., can also be easily migrated.
The Erda API Design Center hosts API documentation in a code repository, a design that associates interface descriptions with interface implementation code. Developers go into the code repository, select the appropriate branch of code, and maintain interface documentation, which is a good way to keep the documentation in sync with new development features. This idea follows the idea that GitOps configuration is code.
Hosting documents in a repository also means that you can collaborate on documents based on branches. When different users write the same document, they simply cut out a new branch from the source branch, edit the document on the new branch, and then merge the branches. The person in charge of different interfaces of the same service can design their own interfaces at any time and merge them back at any time without mutual influence and obstruction.
The API mart
The API mart is an API portal for publishing, archiving, exposing, and debugging apis. The API provider publishes the designed documents to the marketplace for subsequent management and monitoring. API users browse, explore, debug, and subscribe to apis in the API mart. A marketplace is a place where API owners and consumers complete “transactions.”
API mart uses semantic versioning mechanism to achieve API document versioning. The version number is in the format of major.minor.patch, where:
- Major Indicates the major version number. A major version number change usually indicates a major change or an incompatible change.
- Minor indicates the minor version number. A change in the minor version number usually indicates that a new feature has been added but is still backward compatible.
- Patch is a revision number, and a change in the revision number usually means a small, partial revision to an existing version.
In addition to semantic version numbers, there is a version tag called “version name,” which is typically a self-explanatory word or phrase that represents the name of the current version of a document. The version name is unique to the major in the semantic version number and can be thought of as an alias for the major version number. For example, macOS version number Big Sur 11.2.2, where “Big Sur” is the version name of the operating system, “11” is the major version number corresponding to the version name, and “2.2” is the minor version number and revision number respectively. The benefit of versioning in this way is that the growth of API documentation is equated with the growth of the application, allowing you to see the functionality of the application from the API’s perspective. Version numbers explain compatibility and dependencies between service transitions so that both owners and consumers can clearly understand service changes based on version number semantics.
API resources can be associated with specific service instance addresses on the Erda Cloud. Through such association, THE API provider can further implement the API access management, and the caller can also apply to call and test the interface in the API mart.
API Access Management
API access management refers to the management, protection, and auditing of invoked apis when interfaces are exposed to the outside world. Creating access management for API resources is often the time to identify, constrain, and audit client behavior when opening up an API. The access management methods include client authentication and SLA quota management.
After the API provider associates the API resource with the specific service instance address on the Erda Cloud in the marketplace, then creates the access management for the API resource, and the caller can apply to invoke the API in the API marketplace. The provider approves the invocation request and sets the SLA quota for the client. Approved clients gain access qualification and can access the interface externally. After that, callers can also switch API versions in access management and forward requests to service instances corresponding to different versions, thus upgrading or rolling back API versions without the customer being aware of them.
The API access management functionality is built on the capabilities of the Erda cloud native Gateway product, which is much simpler than using the configuration capabilities of the gateway directly — the user only deals with the API.
API lifecycle management best practices
API management products based on the Erda Cloud implement best practices for API lifecycle management. As shown in the figure below, you can view the API from both the provider and caller perspectives.
From the perspective of THE API provider: first of all, we need to update the documents of the API Design center in time to follow the changes of service functions, because the documents are also based on the code repository management, and we can ensure the timely synchronization of THE API documents through the method of code review. In the development tuning phase, the API provider can publish THE API documentation to the marketplace, and other module functions that rely on this interface can be developed in parallel. If the API needs to be opened to the outside world, the API provider will set the access management function for the corresponding API resources. The access management console can observe the external call traffic in real time.
From the API caller’s point of view: If you are a test engineer, you should design automated interface test cases based on the API documentation provided by the developer, rather than maintaining a test-specific interface documentation. If the external integrator discovers the required functional interface through the API marketplace, it should conduct a simple interface access test in the API marketplace after successfully applying for invocation to confirm that the function meets expectations. Then according to the API documents for the integration module code writing, deployment; Finally, you can view the call traffic in My Access.
Software iterates over its life cycle, and so do apis. Both API providers and callers should pay attention to the impact of API iterations. The provider should strictly follow the semantic version mechanism of API mart. When Breaking Change occurs, it should create an independent access management portal for the new Major version, and mark the old version to discard, and guide the caller to use the new version. Callers should keep an eye on subscription notifications for the latest version of the API documentation they are using.
The DevOps feature of the Erda Cloud provides CI/CD capabilities in Cloud native scenarios, and API management should be considered as part of CI/CD as well. You can use the automated test platform of Erda Cloud to connect to the API marketplace and add automated interface testing into the CI process. You can use the pipelined extension of Erda to automatically publish the API version after the CD process and automatically associate the K8s Service address of the Service.
Erda Cloud provides one-stop service for enterprises to implement system architecture based on Cloud native. Erda Cloud API management is also a product naturally grown on the soil of Cloud native. API life-cycle management is a key part of enterprise digitization. If enterprises adopt the cloud native architecture, they must choose the API management products that match it. Otherwise, it may lead to increased adaptation costs and low management efficiency.
Welcome to learn more about the latest information about erda Cloud products on our website, Erda. cloud. Also welcome to our Github repo, our code has been all open source, looking forward to your star ~~~
Welcome to open source
As an open source one-stop cloud native PaaS platform, Erda has platform-level capabilities such as DevOps, micro-service observation governance, multi-cloud management and fast data governance. Click the link below to participate in open source, discuss and communicate with many developers, and build the open source community. Welcome to follow, contribute code and Star!
- Erda Github address:https://github.com/erda-project/erda
- Erda Cloud official website:https://www.erda.cloud/