Brief introduction:Pixiu is a Dubbogo-based cloud-native, high-performance, extensible microservices API gateway. As a gateway product, Pixiu helps users easily create, publish, maintain, monitor, and secure APIs of any size, accept and handle thousands of concurrent API calls, including traffic management, CORS support, authorization and access control, restrictions, monitoring, and API versioning.
Pixiu is what
Before answering what Pixiu is, let’s briefly explain what Dubbo is. Dubbo is an open source high-performance RPC framework with rich service governance capabilities and excellent scalability capabilities. Dubbo extends beyond Dubbo-Go, providing users with Golang’s Dubbo solution, bridging the gap between the two languages and making Dubbo more cloud-native.
Dubbo-Go, as a Golang service, implements mutual calls with the Dubbo service. However, in everyday usage scenarios, users often have a need to expose the Dubbo service in a RESTful style while also balancing internal Dubbo calls. To solve this scenario, the Dubbo API Gateway serves as thePixiu(Chinese: PI xiu, formerly known as Dubbo-Go-Proxy) was born at the historic moment. The reason why we use the name Pixiu is because the image of Zuul, a similar product of Java, is a western monster. As a domestic product, Pixiu uses a similar mythical animal named Pixiu in China as the project name. It also expresses the Dubbo community’s determination to extend a complete cloud native ecosystem.
Currently, Dubbo has a multilingual ecosystem, with Java being the best developed, followed by Golang, and all the other languages lagging behind. Dubbo-Go-Pixiu project is a project developed based on Dubbo-Go. At present, the interface protocol layer supports seven layers of HTTP request invocation. It is planned to support GRPC request invocation in the future 0.5 version. Its other mission is to serve as a new multilingual Dubbo solution.
Why use Pixiu
Pixiu is a Dubbogo-based cloud-native, high-performance, extensible microservices API gateway. As a gateway product, Pixiu helps users easily create, publish, maintain, monitor, and secure APIs of any size, accept and handle thousands of concurrent API calls, including traffic management, CORS support, authorization and access control, restrictions, monitoring, and API versioning. In addition, as a derivative product of Dubbo, Pixiu can help Dubbo users to implement protocol conversion and achieve cross-system and cross-protocol service capability interoperability.
The overall design of Pixiu follows the following principles:
- High Performance: High throughput and millisecond latency.
- Extensible: With the Go-Plugin, users can extend the functionality of Pixiu according to their own needs.
- Easy to use: users with a small amount of configuration, you can go online.
Pixiu features and core functionality
- Support for RESTful APIs and Dubbo APIs
Non-RESTful APIs and Dubbo services often need to be modified before they can be exposed in a RESTful API style. Pixiu provides protocol conversion capabilities. With Pixiu, developers can configure their HTTP API or Dubbo API to be available in a RESTful API style. Version 0.2.1 already supports HTTP-DUBBO protocol conversion based on generalization calls and HTTP protocol forwarding. In future releases, the community will add support for GRPC and HTTP2 protocols.
- User-facing configuration
The configuration of a typical gateway is cumbersome and complex. Pixiu, which aims to be an easy-to-use Gateway product, is designed with three configuration layers: the Gateway global configuration, the API Resource configuration, and the HTTP Verbs method configuration. Through three different levels of configuration, you can achieve both deep customization and support a unified default configuration. At the same time, support local configuration files, can also use the Unified Configuration Server. In addition, a console module is provided, through which hot updates of the configuration are supported. Pixiu’s companion console interface is also under development.
- Integration of common functions
Common functions such as retry, fusing, flow control, and access control no longer need to be implemented repeatedly on each back-end service. With Pixiu, by configuring Filter, developers can have global control and configure their own rules according to the API. This allows developers to focus on business logic and services rather than spending time maintaining the infrastructure.
- extensible
Different usage scenarios have their own unique requirements. To meet the customization needs of different users, Pixiu uses the plug-in pattern. Developers can embed their own unique business logic in the Pixiu gateway in the form of Filter by writing Go Plugin to realize functions such as enterprise login authentication.
Figure 1: A list of core Pixiu functions
The architecture design of Pixiu
支那
Figure 2: The Pixiu architecture
Pixiu: namely Dubbo-Go-Pixiu, consists of four main modules: Listener, Router, Filters and Clients; Dubbo Cluster: A Cluster of Dubbo Services containing one or more Dubbo Services; Other Cluster: the Cluster of services Other than Dubbo, currently supports HTTP services, and will expand to support Other services such as GRPC in the future; Registry Center: a Registry that maintains the invocation address information of each business service; The Metadata Center maintains the configuration information of each business service and stores the configuration information of Pixiu itself.
As an API gateway derived from Dubbo, Pixiu uses Golang to build, mainly because: 1. Golang’s G-M-P, Net Poller and other features make Golang very suitable for building IO intensive applications; 2. 2. Using Golang, you can directly import some of the components in Dubbo-Go, simplifying development.
The entire Pixiu can be roughly broken down into four main modules: Listener, Router, Filters, and Client.
1, the Listener
In the case of Pixiu, the Listener represents how the Pixiu can be accessed externally. Expose the Gateway by configuring properties such as protocol type, address, port, etc. Currently, it supports HTTP protocol, and GRPC will be added in the future.
Listeners: - name: "net/HTTP address: socket_address: protocol_type:" HTTP "address:" 0.0.0.0 "port: 8888 config: idle_timeout: 5s read_timeout: 5s write_timeout: 5s
2, the Router
The Router is the routing component of Pixiu. According to the configuration file, Pixiu stores the exposed URLs in the memory in the form of a tree. When a request is made to the Router component, the corresponding back-end service and its API configuration will be found according to the URL and HTTP methods, and the information will be encapsulated in the request, which is the subsequent filter. And a call to the client to provide enough content.
At this stage, the Router provides the following capabilities:
- Support request one-to-one forwarding routing configuration or wildcard routing configuration.
- Support for forwarding of HTTP requests to a back-end HTTP service.
- Support for HTTP requests to be converted into Dubbo Generalized Invocations requests.
3, Filters,
Filters are the main component of Pixiu that provides additional functionality and extensibility. Its implementation is similar to the filter in Dubbo-Go. According to the filter specified in the configuration, the call chain is generated. Thus, before the back-end service is called, the logic in each filter is run once to achieve throttling, logging and other functions.
If users need a customized filter, they can implement it by writing a GO-plugin. In the configuration, you can load the.so file and specify the plugin group and plugin name implementation in the API config.
PluginFilePath: "" PluginGroup: -groupName: "group1" Plugins: -Name: "rate limit" version: "0.0.1" priority: ExternalLookupName: "ExternalPluginRateLimit" -Name: "Access" version: "0.0.1" priority: -name: "blacklist" version: "0.0.1" priority: 1000 externalLookupName: "ExternalPluginBlackList"
4, Client
The Client is responsible for invoking specific services. Currently, Pixiu supports HTTP and Dubbo back-end services. The community will gradually add other clients such as GRPC to accommodate different protocols.
The implementation of the HTTP Client is relatively simple. The request is generated and invoked through the Golang official package net/ HTTP based on the back-end service information obtained in the Router.
The implementation of the Dubbo Client is slightly more complex than that of the HTTP Client, and is based on the generic invocation of the Dubbo service. Generalized invocation technique is a very basic function provided by Dubbo. You only need to know the method name, parameter type, and return value type of the invocation to initiate a service invocation. The generalized invocation from the client to the server can either discover the service through the registry or connect directly to the server to realize the dynamic invocation of the service.
As shown in the code below, Pixiu dynamically configuring the ReferenceConfig and then using GetRPCService to generate Dubbo’s Generic Client to make the next call.
referenceConfig := dg.NewReferenceConfig(irequest.Interface, context.TODO())
referenceConfig.InterfaceName = irequest.Interface
referenceConfig.Cluster = constant.DEFAULT_CLUSTER
var registers []string
for k := range dgCfg.Registries {
registers = append(registers, k)
}
referenceConfig.Registry = strings.Join(registers, ",")
if len(irequest.DubboBackendConfig.Protocol) == 0 {
referenceConfig.Protocol = dubbo.DUBBO
} else {
referenceConfig.Protocol = irequest.DubboBackendConfig.Protocol
}
referenceConfig.Version = irequest.DubboBackendConfig.Version
referenceConfig.Group = irequest.Group
referenceConfig.Generic = true
if len(irequest.DubboBackendConfig.Retries) == 0 {
referenceConfig.Retries = "3"
} else {
referenceConfig.Retries = irequest.DubboBackendConfig.Retries
}
dc.lock.Lock()
defer dc.lock.Unlock()
referenceConfig.GenericLoad(key)
clientService := referenceConfig.GetRPCService().(*dg.GenericService)
In fact, on the client side of the generalization call, the key step in actually performing the generalization call is the generic\_filter in dubbo-go (the code snippet below). When invoking the Invoke of the generic\_filter, it is agreed that the first parameter list is the method name, the second is the parameter type list, and the third is the parameter value list. The generic\_filter converts the list of parameter values requested by the user into a unified format map (struct2MapAll in the code), which turns the positive deserialization of the class (struct in golang) into the positive deserialization of the map. This allows the generic invocation of the Dubbo service to be accomplished without the POJO description being hard-coded into the Hessain library.
func (ef *GenericFilter) Invoke(ctx context.Context, invoker protocol.Invoker, invocation protocol.Invocation) protocol.Result {
if invocation.MethodName() == constant.GENERIC && len(invocation.Arguments()) == 3 {
oldArguments := invocation.Arguments()
if oldParams, ok := oldArguments[2].([]interface{}); ok {
newParams := make([]hessian.Object, 0, len(oldParams))
for i := range oldParams {
newParams = append(newParams, hessian.Object(struct2MapAll(oldParams[i])))
}
newArguments := []interface{}{
oldArguments[0],
oldArguments[1],
newParams,
}
newInvocation := invocation2.NewRPCInvocation(invocation.MethodName(), newArguments, invocation.Attachments())
newInvocation.SetReply(invocation.Reply())
return invoker.Invoke(ctx, newInvocation)
}
}
return invoker.Invoke(ctx, invocation)
}
conclusion
From the above four modules and a brief introduction to the registry, we can see that when the request is received by the Pixiu via the listener, the request is passed to the router. The Router, based on the configuration of the interface, finds the target back-end service from the original request and sends it to the Filter component along with the associated API configuration. The filter component is executed in sequence according to the original request, API configuration and other information, and finally the request reaches the client, through which the back-end service is called.
The future of Pixiu
Figure 3: Pixiu iteration milestones
In addition to Pixiu as a gateway product, spin-off projects will also be included in our future plans, with the main purpose of providing better usability. For example, due to the lack of native annotations in the Golang language, dubbo-go needs to generate metadata for the service in the form of a configuration file and write it to the registry. Classes related education company classmate wrote a scan code tools \ _https: / / github.com/jack15083/dubbo-go-proxy-tool\_, before each RPC service method combined with the corresponding annotations, thus before the service start scanning through the generate metadata annotation. Pixiu version also plans in the future by providing package, allowing services via annotations \ _https: / / github.com/MarcGrol/golangAnnotations \ _ generated API configuration and register to the Pixiu.
Pixiu is currently positioned as a seven-tier protocol gateway, and its original version was defined as a service gateway for Dubbo. As a product in the cloud era, Pixiu’s development direction is bound to be cloud-oriented. The current version, 0.2.1, has implemented basic Dubbo/ HTTP service proxies and some common gateway functionality. Currently in development, 0.4 and later versions support GRPC and Spring Cloud service invocation, and MQ service support will be provided in the future. In addition, the community will continue to optimize the configuration of the way to reduce the user’s difficulty, continue to optimize the official filter, so that Pixiu can achieve more common gateway functionality at the official level.
Within the next year, the community plans to support the XDS API to evolve Pixiu into a Sidecar for Dubbo Mesh. The ultimate goal is to evolve Proxy Service Mesh form from the existing Dubbo Mesh form. In addition to the technical benefits of Dubbo Mesh, scripting languages such as JS, Python, PHP, Ruby and Perl can also benefit from this configuration.
The ultimate purpose of Pixiu in Dubbo Mesh is to gradually unify the east-west and north-south data surface traffic in Pixiu and make it gradually have the capability of Application Runtime as a key solution for Dubbo’s multilingual ecology.
The Dubbogo community [No. 23331795] is with Dubbogo.
Zhenyu Feng, Apache Dubbo Committer, is currently managing an entire team in the IT department of a consumer goods company in Hong Kong. Joining the Dubbogo community after accidentally reading an article about Dubbogo in the summer of 2020, he is currently leading the development of Pixiu 0.4.0. \ _
Click https://developer.aliyun.com/community/cloudnative, more cloudy native content!
Copyright Notice:The content of this article is contributed by Aliyun real-name registered users, and the copyright belongs to the original author. Aliyun developer community does not own the copyright and does not bear the corresponding legal liability. For specific rules, please refer to User Service Agreement of Alibaba Cloud Developer Community and Guidance on Intellectual Property Protection of Alibaba Cloud Developer Community. If you find any suspected plagiarism in the community, fill in the infringement complaint form to report, once verified, the community will immediately delete the suspected infringing content.