Wen | Cao Xiansheng, e Sign Bao middleware development
Responsible for the development and maintenance of E Sign Up Middleware, including MQ, gateway, microservice, data synchronization, full-link pressure measurement, etc
Preface by Contributor
“Open source is a process of growing together in use.”
From studying some open source projects of SOFAStack in 2018, to now deeply using MOSN, with SOFA coming to its fourth anniversary.
Because of my interest, I have contacted many open source communities, but I have experienced many components of SOFA community, such as SOFAArk, SOFARPC and MOSN. Years of wandering in the nail group to ask questions, can get timely reply, which is of great help to our research MOSN. Therefore, THROUGH MOSN code design, I learned a lot about Sidecar design concept.
Our starting point for using MOSN is that the corporate framework uses a lot of middleware, and each middleware has its own dependencies that are often in conflict. Although we use Pom management mechanism similar to Spring Boot, jar conflicts will inevitably occur in the process of upgrading the framework if some students introduce JAR packages by themselves. In order to solve this problem, we investigated many solutions and finally decided that Service Mesh was a suitable solution to solve this problem.
At the same time, we also investigated some other open source products. After internal discussion and various trade-offs, we chose MOSN.
When MOSN is used, dynamic routing is required to connect to Eureka. However, there are not many articles about routing on the official website. Therefore, in their own and strong yuan teacher learning, summed up such a route to share the article.
MOSN is a network edge proxy component. The routing function is the core function. This article describes how to use MOSN routing and some advanced skills of using MOSN routing.
Basic Routing Design
In MOSN routing design, Cluster and Route are highly correlated. The configuration of Route is to express how to accurately find the Cluster you want to find. Another Cluster can have multiple Host machines.
For example, a Cluster has 100 machines, of which 50 are V1 and 50 are V2. How can requests be accurately routed to V1 or V2 according to certain rules?
For example, if I want to find the Cluster based on a value in the Header and then calculate this value with a value in the configuration center, how should I configure it?
- First, let’s look at the simplest route setup.
The figure above shows a simple Json configuration. The configuration of Cluster Manager and Routers is the key to routing. We can configure multiple clusters according to the Cluster Manager, with multiple hosts per Cluster.
The MOSN is then told how to route requests to the Cluster according to some rules in the Routers configuration.
The diagram below:
This configuration says that you now have a Rouer configuration named Server_Router, a virtual host that can be configured with multiple domain names, all of which match.
At the same time, there are multiple routing configurations for this domain name. For the moment, there is a routing configuration: prefix matching, which starts with /, is forwarded to the Host in ServerCluster, which is the following Cluster Manager configuration.
In this way, a simple MOSN route configuration is implemented.
Dynamic routing Cluster
In most cases, if our routing logic is simple, such as finding a Cluster based on a name in the Header, the code or configuration will look something like this:
Router := v2. router {// header Match RouterConfig: v2.RouterConfig{Match: v2.RouterMatch{Headers: []v2.HeaderMatcher{app.Name cluster. {Name: "x-service-id ", Value: App. Name,},},}, // Cluster Name matches. Route: v2.RouteAction{ RouterActionConfig: v2.RouterActionConfig{ ClusterName: app.Name, }, }, }, } r.VirtualHosts[0].Routers = append(r.VirtualHosts[0].Routers, router)Copy the codeIf the Header contains the x-service-id kv, then we can find the Cluster for the RouteAction.
What about more complicated logic?
How do you find the Cluster, for example, using the Header in the request and some value in the “configuration center”?
In this case, configuration cannot solve this requirement because it involves computing logic. The MOSN supports this requirement through dynamic configuration.
Configuration as shown below:
We set up a KV configuration (“Cluster_Variable”: “my-Clustervariable”).
At the same time, we also need to use the variable mechanism in StreamFilter to set the Value of key to “my-clustervariable”, which is the calculated Cluster name.
The code is as follows:
// Register this key in the table. func init() { variable.Register(variable.NewStringVariable("My-ClusterVariable", nil, nil, variable.DefaultStringSetter, 0)) } var clusterMap = make(map[int]string, 0) func (f *MyFilter) OnReceive(ctx context.Context, headers api.HeaderMap, buf buffer.IoBuffer, StreamFilterStatus; StreamFilterStatus; StreamFilterStatus; StreamFilterStatus; StreamFilterStatus; StreamFilterStatus := len(clusterMap); This key must be consistent with that in the configuration file. variable.SetString(ctx, "My-ClusterVariable", cluster) return api.StreamFilterContinue }Copy the codeMOSN Subset
As mentioned above, we often have multiple versions in a cluster. How do we route requests to specific versions based on certain tags?
Usually, we use a Subset. Each application can be marked in a Cluster. At the same time, we also configure the related configuration (MOSN is called Metadata) to achieve more complex routes.
The official DOCUMENTATION of the MOSN briefly describes how to use Metadata.
Let’s introduce the use of Subset in more detail:
The figure shows the Cluster Host configuration on the left and the Router configuration on the right.
If Name and Value are specified in the Header and their values Match Service and service. Green, the request is partitioned to Cluster_Subset.
The cluster may have multiple machines, so the metadata of the machine must be the same as the metadata of the route configuration, and all of them must be Subset:Green to match the Host. Otherwise, the system cannot find the Host (the fall_back_policy policy is 0).
Thus, we solve the routing problem of multiple versions of hosts in a Cluster.
Furthermore, a Cluster may have multiple hosts, and each Host may have different subsets, which may require a lot of routes. If all of them are written in configuration files, it is troublesome.
The MOSN supports dynamic routing based on the Stream filter.
As follows:
Based on the variable mechanism of MOSN, the KV Metadata combination is set in the request-level VarRouterMeta, and the effect is similar to the configuration file above.
In addition, if Metadata is configured in the routing configuration, Metadata is configured at the request level. MOSN, then, will merge two metadata, and Host matching, the logical PKG/proxy/downstream. Go: 1497 in the code.
Let’s take a simple example, such as specifying machine calls in groups:
1. Request: Specify an IP address in the Header and set the IP address in VarRouterMeta
2.Host configuration: Configure IP KV in Metadata, for example, 192.168.2.3
The diagram below:
This will match the specified machine.
ps: For this example, we can also use the ORIGINAL_DST mechanism of MOSN to set the Cluster Type to ORIGINAL_DST (MOSN also supports DNS Cluster Type). Then set cluster.original_dST_lb_config.use_header = true. When we make a request, we add Host = {destination address} to the Header, and the MOSN will forward based on the specified Host Header.
Of course, the MOSN can also have a custom name, which does not have to be Host.
To take a complicated example, consider a scenario where a single Host exists in multiple groups and only one group can be specified on a request.
The diagram below:
We now have 2 machines in 3 groups: AAA, BBB and CCC. Each machine contains AAA groups. There are now three requests, each of which is a different grouping.
How do we configure metadata at this point?
First, grouping machines is essentially marking, and we can think of metadata as a Tag list.
The code above shows that we convert multiple grouping labels to kv, a metadata that the MOSN can recognize. Each label has a fixed value true. Value itself has meaning in the MOSN SubsetLB, that is, it finally matches Subset host entry that meets the conditions according to the metadata value carried in the SubsetLB. But since metadata is a map, and because of the specificity of our example, only the key itself can be grouped, all values remain the same, essentially any value is acceptable. Note that these keys must be stored in SubsetSelectors; otherwise, the MOSN cannot identify them. On each call, we extract the grouping label from the Header in Filter and set it to context variables.
Such as:
In this way, we can accomplish more complex packet routing.
So how does MOSN look for Subset?
The code is as follows:
When performing chooseHost subsetLoadBalancer. FindSubset function according to the current request of metadata, to find matching from the subsetLoadBalancer Host List.
conclusion
We started with a simple configuration to implement simple Router and Cluster configuration file routing.
The stream filter can be used to dynamically find clusters. MOSN supports Subset, which can match routing and Cluster Host based on the Route configuration file. If the logic is complicated, it can also dynamically search for Subset based on the Stream filter + varRouterMeta variable.
In most cases, we can solve our routing problems with Json configuration. If it is complicated, we use stream filter + varRouterMeta/Stream filter + Cluster_variable to solve our requirements.
Let’s try to end this article with a diagram
“References”
[1] Router configures MOSN SubsetLB to Load Balancer Subsets
[2] Metadata usage
Recommended Reading of the Week
BabaSSL releases 8.3.0 | requirements for implementing corresponding private computing
HAVE FUN | SOFARegistry source code parsing
BabaSSL: supports semi-homomorphic encryption algorithm EC-Elgamal
Congratulations to Lv Bingjie on becoming a SOFAStack Committer!