Introduction: This article will be introduced more than the “how to use 20 minutes can get the same enterprise-class full link gray capacity?” To further introduce the full link gray scale of the message scenario.

Author: Yi Zhan

In the previous series of articles, we introduced the MSE scenario for full-link traffic control through full-link Canary publishing. We have seen how full-link grayscale should be implemented for RPC calls like Spring Cloud and Dubbo. However, it does not involve the flow control of asynchronous scenarios such as messages. Today, we will introduce the above “how to get the same enterprise-class full-link grayscale capability in 20 minutes”. To further introduce the full link gray scale of the message scenario.

Although most business scenarios do not have such strict requirements on the gray level of messages as RPC, the following two scenarios still have certain demands on the full link of messages.

1, the first scenario is, in the message consumption may produce new RPC calls, if not before the news to follow this ring set full link traffic control rules, can lead to this part of the generated by message flow “escape”, leading to link all rules of gray is destroyed, resulting in not in line with expectations.

To prevent this, we need to revert the original traffic in the message at consumption time and follow the original rules when RPC calls are made. We through the architecture diagram to a more detailed description, meet after this logic, what is call link, we can see from the chart, gray and baseline environment to produce news, although the news push is random, but in the process of consumption, generate new RPC calls, or to return to the flow originally belonging to the environment.

2. The second scenario requires more stringent message grayscale isolation. For example, when the message consumption logic is modified, it is hoped to verify the correctness of the new message consumption logic by means of small traffic. It is strictly required that gray messages can only be pushed to gray message consumers.

Today we will practice the full link grayscale of the second scenario messages. Currently, MSE only supports the grayscale of RocketMQ messages. If you are using RocketMQ open Source version 4.5.0 or above, if you are using RocketMQ Ali Cloud Commercial version, platinum version with Ons Client version 1.8.0.final or above. If you only want to use the first scenario, you only need to enable the full-link grayscale function for APPLICATION B, and there is no need to do additional configuration related to message grayscale.

In this best practice operation, we will deploy the application in Aliyun container service Kubernetes version, that is, ACK cluster to demonstrate, but in fact, message gray scale for the deployment mode of the application is not restricted, you can refer to the MSE help document, find the deployment mode corresponding to their own access. You can also use message full link gray scale.

The premise condition

  1. To open MSE Professional edition, see Opening MSE Microservice Governance Professional Edition **[1]**.
  2. To create an ACK cluster, see Creating a Kubernetes Cluster **[2]**.

steps

Step 1: Access MSE microservice governance

1. Install mSE-ACK-Pilot

  1. Log in to the Container Services console **[3]**.
  2. On the left navigation bar, clickMarketplace > App Catalog.
  3. Click on the Apps catalog pageAliyun Application, the choice ofMicro service, and click theack-mse-pilot.
  4. In the cluster list on the right of the ACK-MSE-Pilot page, select a cluster and click Create.

It takes about 2 minutes to install the MSE microservice governance component.

After the Helm is created, the system automatically switches to the Helm page of the target cluster to check the installation result. If the following page displays related resources, the installation is successful.

2. Enable MSE microservice governance for applications in the ACK namespace

  1. Log in to the MSE Governance Center console **[4]**. If you have not enabled MSE microservice governance, please enable it as prompted.

  2. On the left navigation bar, selectMicroservice Governance Center > Kubernetes Cluster list.
  3. On the Kubernetes Cluster list page, select the cluster name or cluster ID in the search box list, then enter the corresponding keyword, and click the search icon.
  4. Click in the operation column of the target clustermanagement.
  5. In the namespace list area on the cluster details page, click in the operation column of the target namespaceEnable micro-service governance.
  6. In the enable Micro Service governance dialog box, click Ok.

Step 2: Restore the online scene

First, we will deploy four business applications, spring-Cloud-Zuul, Spring-Cloud-A, Spring-Cloud-b and Spring-Cloud-c respectively. And registry Nacos Server and messaging service RocketMQ Server, simulating a real call link.

In the following figure, calls between applications include both Spring Cloud calls and Dubbo calls, covering the two most commonly used microservices frameworks on the market. Application C produces A RocketMQ message, which is consumed by application A. When A consumes the message, it also initiates A new call. These applications are standard uses of Spring Cloud, Dubbo, and RocketMQ in their simplest form, and can also be found directly in the

Github.com/aliyun/alib…

Project view source code.

Before deploying, take a quick look at the call link

When the spring-Cloud-Zuul application receives A “/A/dubbo” request, it forwards the request to Spring-Cloud-A, which then accesses Spring-Cloud-B through the Dubbo protocol. Spring-cloud-b also accesses Spring-Cloud-c through the Dubbo protocol. Spring-cloud-c produces a message and returns its own environment label and IP upon receiving the request. These produced messages are consumed by the Spring-Cloud-A application. When the spring-Cloud-A application consumes the messages, it calls B through spring Cloud, which in turn calls C through Spring Cloud. And output the results to your own log.

When we call /A/dubbo, the return value is A[10.25.0.32] -> B[10.25.0.152] -> C[10.25.0.30] Output log below the 2021-12-28 10:58:50. 301 INFO 1 - [essageThread_15] C.A.M se. Demo. Service. MqConsumer: Topic: TEST_MQ, producer: C [10.25.0.30], invoke the result: A [10.25.0.32] - [10.25.0.152] - > B > C [10.25.0.30]Copy the code

Once you are familiar with the call link, you can deploy the application using Kubectl or directly using the ACK console. The yamL files used for deployment are as follows, which you can also use directly in the

Github.com/aliyun/alib…

Obtain the corresponding source code.

Nacos Server apiVersion: apps/v1 kind: Deployment metadata: name: nacos-server spec: selector: matchLabels: app: nacos-server template: metadata: annotations: labels: app: nacos-server spec: containers: - env: - name: MODE value: "standalone" image: registry.cn-shanghai.aliyuncs.com/yizhan/nacos-server:latest imagePullPolicy: IfNotPresent name: nacos-server ports: - containerPort: 8848 --- apiVersion: v1 kind: Service metadata: name: nacos-server spec: type: ClusterIP Selector: app: nacos-server Ports: -name: HTTP port: 8848 targetPort: 8848 # Deploy service application -- apiVersion: apps/v1 kind: Deployment metadata: name: spring-cloud-zuul spec: selector: matchLabels: app: spring-cloud-zuul template: metadata: annotations: msePilotCreateAppName: spring-cloud-zuul labels: app: Spring-cloud-zuul spec: containers: - env: - name: JAVA_HOME value: /usr/lib/jvm/java-1.8-openJDK/jre-name: The enable. Mq. Invoke the value: 'true' image: registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-zuul:1.0.0 imagePullPolicy: Always name: spring-cloud-zuul ports: - containerPort: 20000 --- apiVersion: v1 kind: Service metadata: annotations: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: slb.s1.small service.beta.kubernetes.io/alicloud-loadbalancer-address-type: internet name: zuul-slb spec: ports: - port: 80 protocol: TCP targetPort: 20000 selector: app: spring-cloud-zuul type: LoadBalancer status: loadBalancer: {} --- apiVersion: apps/v1 kind: Deployment metadata: name: spring-cloud-a spec: selector: matchLabels: app: spring-cloud-a template: metadata: annotations: msePilotCreateAppName: spring-cloud-a labels: app: spring-cloud-a spec: Containers: - env: -name: JAVA_HOME value: /usr/lib/jvm/java-1.8-openJDK /jre image: Registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-a:1.0.0 imagePullPolicy: Always name: spring -- cloud - a ports: - containerPort: 20001 livenessProbe: tcpSocket: port: 20001 initialDelaySeconds: 10 periodSeconds: 30 --- apiVersion: apps/v1 kind: Deployment metadata: name: spring-cloud-b spec: selector: matchLabels: app: spring-cloud-b template: metadata: annotations: msePilotCreateAppName: spring-cloud-b labels: app: spring-cloud-b spec: containers: - env: - name: JAVA_HOME value: /usr/lib/jvm/java-1.8 -openJDK /jre image: Registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-b:1.0.0 imagePullPolicy: Always name: spring -- cloud - b ports: - containerPort: 20002 livenessProbe: tcpSocket: port: 20002 initialDelaySeconds: 10 periodSeconds: 30 --- apiVersion: apps/v1 kind: Deployment metadata: name: spring-cloud-c spec: selector: matchLabels: app: spring-cloud-c template: metadata: annotations: msePilotCreateAppName: spring-cloud-c labels: app: spring-cloud-c spec: containers: - env: - name: JAVA_HOME value: /usr/lib/jvm/java-1.8 -openJDK /jre image: Registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-c:1.0.0 imagePullPolicy: Always name: spring -- cloud - c ports: - containerPort: 20003 livenessProbe: tcpSocket: port: 20003 initialDelaySeconds: 10 periodSeconds: 30 --- apiVersion: apps/v1 kind: Deployment metadata: name: rockectmq-broker spec: selector: matchLabels: app: rockectmq-broker template: metadata: labels: app: rockectmq-broker spec: containers: - command: -sh-mqbroker - '-n' - 'mqNAMesrv :9876' - '-c /home/rocketmq-rocketmq-4.5.0 /conf/broker.conf' env: -name: ROCKETMQ_HOME value: / home/rocketmq/rocketmq - 4.5.0 image: Registry.cn-shanghai.aliyuncs.com/yizhan/rocketmq:4.5.0 imagePullPolicy: Always name: rockectmq - broker ports: - containerPort: 9876 protocol: TCP - containerPort: 10911 protocol: TCP - containerPort: 10912 protocol: TCP - containerPort: 10909 --- apiVersion: apps/v1 kind: Deployment metadata: name: rocketmq-name-server spec: selector: matchLabels: app: rocketmq-name-server template: metadata: labels: app: rocketmq-name-server spec: Containers: -command: -sh-mqnamesrv env: -name: ROCKETMQ_HOME value: /home/rocketmq-rocketmq-4.5.0 image: Registry.cn-shanghai.aliyuncs.com/yizhan/rocketmq:4.5.0 imagePullPolicy: Always name: rocketmq - name - server ports: - containerPort: 9876 protocol: TCP - containerPort: 10911 protocol: TCP - containerPort: 10912 protocol: TCP - containerPort: 10909 protocol: TCP --- apiVersion: v1 kind: Service metadata: name: mqnamesrv spec: type: ClusterIP selector: app: rocketmq-name-server ports: - name: mqnamesrv-9876-9876 port: 9876 targetPort: 9876Copy the code

If the installation is successful, the following is an example:

➜ ~ kubectl get SVC,deploy NAME TYPE cluster-ip external-ip PORT(S) AGE service/kubernetes ClusterIP 192.168.0.1 <none> 443/TCP 7D service/ mqNAMesrv ClusterIP 192.168.213.38 < None > 9876/TCP 47H service/ nacOS-server ClusterIP 192.168.24.189 < None > 8848/TCP 47h service/zuul-slb LoadBalancer 192.168.189.111 123.56.253.4 80:30260/TCP 47h NAME READY up-to-date AVAILABLE AGE deployment.apps/nacos-server 1/1 1 1 4m deployment.apps/rockectmq-broker 1/1 1 1 4m deployment.apps/rocketmq-name-server 1/1 1 1 5m deployment.apps/spring-cloud-a 1/1 1 1 5m deployment.apps/spring-cloud-b  1/1 1 1 5m deployment.apps/spring-cloud-c 1/1 1 1 5m deployment.apps/spring-cloud-zuul 1/1 1 1 5mCopy the code

We can also verify the call link with zuul-SLB

➜ ~ curl http://123.56.253.4/A/dubbo A [10.25.0.32] - [10.25.0.152] - > B > C [10.25.0.30]Copy the code

Step 3: Enable the message gray function

Now, as prompted by the console, turn on the message’s grayscale on both spring-Cloud-C, the message’s producer, and Spring-Cloud-A, the message’s consumer. Let’s open it directly from the MSE console, click to go to the app details page, and select the “Message Grayscale” TAB.

As you can see, in the tag ignored by the unmarked environment, we entered gray, which means that the message with the flag of the Gray environment can only be consumed by Spring-Cloud-a-Gray, not by Spring-Cloud-a.

1. An extra note is needed here. Considering that in the actual scenario, the owner of spring-Cloud-C application and Spring-Cloud-A application may not be the same person, and they may not be able to carry out the grayscale release synchronization operation at the same time, The default behavior of an unmarked environment is to consume all messages. In this way, spring-Cloud-C does not need to force the spring-Cloud-A application to be grayscale published at the same time when performing grayscale publishing.

2. We give the owner of Spring-Cloud-A the option to consume the unlabeled environment. If you want to implement the unlabeled environment not to consume the messages produced by C-Gray, you only need to configure it in the console and the configuration takes effect in real time.

  • You do not need to modify the application code or configuration to use this feature.
  • Message producers and message consumers need to enable message grayscale at the same time for the message grayscale function to take effect.
  • Message types currently only support RocketMQ, including the open source version and Ali Cloud Business edition.
  • If you use open source RocketMQ, both RocketMQ Server and RocketMQ Client need to use version 4.5.0 or later.
  • If you use Aliyun RocketMQ, you need to use the platinum version, and Ons Client uses 1.8.0.Final or later.
  • After greyscale is turned on, MSE modifies the message’s Consumer Group. For example, the original Consumer Group is group1 and the environment label is gray. After the message gray function is enabled, the Group will be changed to group1_gray. If you use RocketMQ, create the Group in advance.
  • SQL92 filtering is used by default. If you are using open source RocketMQ, you will need to enable this feature on the server (that is, enablePropertyFilter=true in broker.conf).
  • By default, unmarked nodes consume messages from all environments. If you want to specify that unmarked links do not consume messages generated by a label environment, configure Labels ignored by unmarked environments. The configuration takes effect dynamically without restarting the application.

Step 4: Restart the node, deploy the application of the new version, and import traffic for verification

First of all, after enabling and disabling the message grayscale function of the application, the node needs to be restarted to take effect. Therefore, we need to restart the spring-Cloud-a and Spring-Cloud-c applications first, which can be redeployed on the console. Or simply use the kubectl command to delete the existing pod.

Then, go ahead and deploy the new versions of Spring-Cloud-a-Gray, spring-Cloud-b-Gray, and Spring-Cloud-c-Gray in the Kubernetes cluster using yamL files

apiVersion: apps/v1 kind: Deployment metadata: name: spring-cloud-a-gray spec: selector: matchLabels: app: spring-cloud-a-gray template: metadata: annotations: alicloud.service.tag: gray msePilotCreateAppName: spring-cloud-a labels: app: spring-cloud-a-gray spec: containers: - env: - name: JAVA_HOME value: The/usr/lib/JVM/Java - 1.8 - its/jre image: registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-a:1.0.0 imagePullPolicy: Always name: spring-cloud-a-gray ports: - containerPort: 20001 livenessProbe: tcpSocket: port: 20001 initialDelaySeconds: 10 periodSeconds: 30 --- apiVersion: apps/v1 kind: Deployment metadata: name: spring-cloud-b-gray spec: selector: matchLabels: app: spring-cloud-b-gray template: metadata: annotations: alicloud.service.tag: gray msePilotCreateAppName: spring-cloud-b labels: app: spring-cloud-b-gray spec: containers: -env: -name: JAVA_HOME value: /usr/lib/jvm/java-1.8 -openJDK /jre image: Registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-b:1.0.0 imagePullPolicy: Always name: spring - the cloud - b - gray ports: - containerPort: 20002 livenessProbe: tcpSocket: port: 20002 initialDelaySeconds: 10 periodSeconds: 30 --- apiVersion: apps/v1 kind: Deployment metadata: name: spring-cloud-c-gray spec: selector: matchLabels: app: spring-cloud-c-gray template: metadata: annotations: alicloud.service.tag: gray msePilotCreateAppName: spring-cloud-c labels: app: spring-cloud-c-gray spec: containers: - env: - name: JAVA_HOME value: The/usr/lib/JVM/Java - 1.8 - its/jre image: registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-c:1.0.0 imagePullPolicy: Always name: spring-cloud-c-gray ports: - containerPort: 20003 livenessProbe: tcpSocket: port: 20003 initialDelaySeconds: 10 periodSeconds: 30Copy the code

Once the deployment is complete, we import the traffic and verify it

  1. Log in to the MSE Governance Center console **[4], the choice ofList of applications **.
  2. Click Apply Spring-Cloud-aApplication detailsMenu, you can see that all traffic requests are to the unmarked version of the Spring-Cloud-A application, i.e. the stable version.

  1. Click the Add button in the TAB Route at the bottom of the page to set the gray rule for the Gray version of the Spring-Cloud-A application.

  1. Initiate the flow call, we through Zuul-SLB, respectively initiate the flow call, and check the gray level of the situation.

We looked at message consumption through the spring-Cloud-a and Spring-Cloud-a-Gray logs. As you can see, the message grayscale function is in effect. The spring-Cloud-a-Gray environment will only consume messages with gray labeled, and the Spring-Cloud-A environment will only consume messages produced by unlabeled traffic.

TEST_MQ, producer: Cgray [10.25.0.102], invoke result: Agray[10.25.0.101] -> Bgray[10.25.0.25] -> Cgray[10.25.0.102], spring-cloud-a-Gray will only consume messages produced by Cgray, In addition, the Spring Cloud call initiated in the process of consuming the message results in Agray[10.25.0.101] -> Bgray[10.25.0.25] -> Cgray[10.25.0.102], that is, closed loop in the gray environment.

And the spring-Cloud-A environment, Topic :TEST_MQ,producer:C[10.25.0.157],invoke result:A[10.25.0.100] -> B[10.25.0.152] -> C[10.25.0.157], Only messages produced by C’s baseline environment are consumed, and Spring Cloud calls made during this process are also closed loop in the baseline environment.

Step 5: Adjust label filtering rules for messages and verify them

Considering that the owner of the Spring-Cloud-C application and the spring-Cloud-A application may not be the same person in the actual scenario, it may not be possible to synchronize the grayscale publishing of the two applications at the same time. Therefore, in the grayscale of messages, the default behavior of the unmarked environment is to consume all messages. In this way, spring-Cloud-C does not need to force spring-Cloud-A applications to be grayscale published at the same time and use the same environment standard when performing grayscale publishing.

When spring-Cloud-A consumes, the owner of Spring-Cloud-A has the right to choose the behavior of the unmarked environment. If the unmarked environment does not consume the messages produced by C-Gray, it only needs to be configured in the console, and the configuration takes effect in real time.

  1. Adjust the filtering rules for spring-Cloud-A unmarked environment. For example, to select that the unlabeled environment will no longer consume messages produced by the Gray environment, simply select Gray in the “Labels ignored by the unlabeled environment” box and click OK.

  1. After adjusting the rules, the rules can take effect dynamically without restarting. We can directly view the logs of Spring-Cloud-A to verify that the rule adjustment takes effect.

From this log, we can see that the baseline environment can consume messages produced by gray and baseline at the same time, and Spring Cloud calls generated when consuming the corresponding environment messages are routed to gray and baseline respectively.

Operation summary

  1. The entire process of full-link message graying does not require any code or configuration changes.
  2. Currently, RocketMQ is only supported, and Client versions need to be later than 4.5.0. The RocketMQ Server must support SQL92 rule filtering, that is, the open-source RocketMQ must be set to enablePropertyFilter= True, and the Ali Cloud RocketMQ must use the platinum version.
  3. After the message gray level is enabled, MSE Agent will modify the group of message consumers. For example, the original consumption group is group1 and the environment label is gray, the group will be changed to group1_gray. If RocketMQ is used, You need to create the modified group in advance.
  4. After turning on or off message gray scale, the application needs to be restarted to take effect. The function of modifying labels ignored in an unmarked environment takes effect dynamically and does not require a restart.

A link to the

[1] MSE Microservice Governance Professional Edition:

Help.aliyun.com/document_de…

[2] Kubernetes cluster:

Help.aliyun.com/document_de…

[3] Container Service Console:

cs.console.aliyun.com/

[4] MSE Governance Center console

mse.console.aliyun.com/#/msc/home

The original link

This article is the original content of Aliyun and shall not be reproduced without permission.