1. Microservices

Microservices have been very hot and talked about in the last couple of years. In simple terms, microservices are a way of developing a single application as a suite of small services, each of which has its own processes and uses lightweight mechanisms (usually HTTP source apis) to communicate. Here’s an example:

Microservices Architecture:

The two figures above have visualized what microservices are, and the way software is deployed needs to be built on containers. The microservice-related ecosystem will be mature in The Java and Go languages, especially Java. Python as the back end, this aspect will be relatively weak, microservice framework can be seen only Nameko, and the technology is not so mature, because the current business Python scene is a little more than Go language, so let’s play Python to play microservice.

2. Service Mesh and Serverless

There are two other concepts that are also hot on the micro services front, which are briefly mentioned below.

2.1 Service Mesh

The concept of Service Mesh is obscure at first. There is also talk about the next generation of microservices online. Simply put, when thousands of microservices are deployed on Kubernetes, the overall complexity is quite complex because each microservice requires health checks, error handling, latency, etc. While Kubernetes can provide health check and automatic recovery, but also need to fuse mode, service discovery, API management, encryption verification and so on. These are the problems that Service Mesh needs to solve.

More detailed introduction: philcalcado.com/2017/08/03/…

As an infrastructure layer for communication between services, it can be likened to TCP/IP between applications or microservices, responsible for network calls, traffic limiting, fuses, and monitoring between services. The TCP/IP layer is generally not a concern for writing applications (such as RESTful applications over HTTP), and the use of Service Mesh eliminates the need for relationships between services that are previously implemented by applications or other frameworks, such as The Spring Cloud. Now we just hand it over to the Service Mesh.

Service Mesh has the following characteristics:

An intermediate layer of communication between applications
Lightweight Web proxy
The application is not aware
Decouple application retry/timeout, monitoring, tracing, and service discovery

The architecture of the Service Mesh is as follows:

Popular open source software is Istio, so try it again.

2.2 Serverless

The non-service architecture, first introduced to at AWS tech Summit, simply means that you don’t care about the server, and the entire computing stack, including the operating system processes that run the functional code, is managed entirely by the cloud provider. It reinforces the idea of DevOps.

Having actually played with the AWS Lambda unserved application, it’s really convenient to simplify it to a function that implements Web services through API Gateway + Lambda. Charging per request is a bit of a pitfall at present, especially when the number of requests is large, which is far more expensive than deploying your application on Docker. Therefore, the current scenario without service architecture is also very suitable for some one-time tasks, the request volume is not much to call the scene will be very convenient, the developer members can develop their own deployment, no longer need to care about the server. All operations can be eliminated, and developers can focus more on core business development to achieve rapid roll-out and iteration.

3. Python framework for building microservices

3.1 Nameko Introduce

Nameko is the micro service framework in Python, git to https://github.com/nameko/nameko, popularity is not high, yet the introduction of the official document has realized:

It comes with built-in support for:

RPC over AMQP
Asynchronous events (pub-sub) over AMQP
Simple HTTP GET and POST
Websocket RPC and subscriptions (experimental)

Simply speaking, RPC is based on AMQP, which implements publish subscription, simple HTTP service and Websocket RPC.

This feels like child’s play to the Java ecosystem. The architecture uses RabbitMQ as a Message broker for communication between Nameko services.

See the official documentation for more details.

3.2 Practice

Let’s take a business scenario as an example.

Scenario: Suppose that in a social scenario, when commenting on other people’s posts, the server pushes a message to the post author to let them know that there are comments, and the comments must be registered first.

It involves two micro-services, registration service and push service, as well as a comment interface.

3.2.1 Environment Construction:

Python3.5 +
RabbitMQ
Redis 3.2.1
Nameko 2.11.0
Swagger
Flask 1.0.2

The Python3 environment needs to be prepared first. Redis is the storage for user login registration for simplicity, Nameko is installed with PIP, and RabbitMQ is best installed with Docker.

RabbitMQ docker installation commandDocker search rabbitmq docker pull RabbitMQ: 3.7-RC-management Docker run-d --hostname my-rabbit --name some-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3.7-rc-management
Need to run on port 5672 by default

Doc: https://github.com/docker-library/docs/tree/master/rabbitmq

# nameko run the service command:Nameko run service --broker amqp://guest:guest@localhost where guest:guest is the username and password of the RabbitMQ Docker imageCopy the code

To facilitate API testing, flasgger provides Swagger UI to integrate Flask.

With the environment ready, start the code section of the demo.

3.2.2 Code Demonstration:

├ ─ ─ app │ └ ─ ─ API. Py ├ ─ ─ dependence │ ├ ─ ─ just set py │ └ ─ ─ services │ ├ ─ ─ just set py │ ├ ─ ─ config. Py │ └ ─ ─ │ ├─ push │ ├─ pressCopy the code

The code structure is as follows:

App stores API interface services.
Dependence can be understood as a basic module. Maybe a lot of microservices rely on encapsulated services, such as redis and mysql interface. If Git warehouse is generally used, submodule can be used to the warehouse of specific service.
Microservices code, here demonstrates two services, registration and push services.

There are two features that need to be implemented:

How to invoke microservices in API code
How can other microservices be invoked in microservices

Here’s the code from dependence:

# content of redis_service
class RedisService(object):
    def __init__(self):
        self.redis_instance = RedisClient.get_redis(
            config.REDIS_NAME, config.REDIS_HOST, config.REDIS_PORT,
            config.REDIS_DB)
        self.users_key = "users"
        self.users_data_key = "users_data"

    def check_registered_and_get_info(self, u_id):
        """ Check if the user is registered and return user information if registered. """
        user_data = self.redis_instance.hget(self.users_data_key, u_id)
        if not user_data:
            return False.None
        return True, json.loads(user_data)

    def check_email_is_registered(self, email):
        u_id = self.redis_instance.hget(self.users_key, email)
        return u_id

    def register(self, u_id, email, data):
        self.redis_instance.hset(self.users_key, email, u_id)
        result = self.redis_instance.hset(self.users_data_key, u_id,json.dumps(data))
        return result
Copy the code

Check_registered_and_get_info Check_registered_and_get_info Check_registered_and_get_info Check_registered_and_get_info Check_registered_and_get_info
Check_email_is_registered Checks whether the mailbox is registered twice
Register Registers and stores user information

Let’s look at the API code:

# content of api.py
import time
import random
from flask import Flask, request, jsonify
from flasgger import Swagger
from nameko.standalone.rpc import ClusterRpcProxy
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--port", help="app running port", type=int, default=5000)
parse_args = parser.parse_args()

app = Flask(__name__)
Swagger(app)

CONFIG = {'AMQP_URI': "amqp://guest:guest@localhost"}
Copy the code

Nameko is the RPC service proxy that provides microservices. It also needs to provide CONFIG. The content is Message Broker address, which is actually RabbitMQ. Swagger is combined with Flask to facilitate API testing in a web interface.

# content of api.py
@app.route('/api/v1/comment', methods=['POST'])
def comment(a):
    """ Comment API Parameters Explain: Timestamp Comment time u_id User ID Content Comment content article_id Article ID article_u_id Author ID parent_comment_id Parent comment ID (optional) -- parameters: - name: body in: body required: true schema: id: comment properties: timestamp: type: integer u_id: type: string content: type: string article_id: type: integer article_u_id: type: integer parent_comment_id: type: integer responses: code: description: 0 Comment Success! message: description: Error Message! data: description: return comment_id """
    data = request.json
    article_u_id = data.get("article_u_id")
    u_id = data.get("u_id")
    code, message = 0.""
    if not article_u_id or not u_id:
        code, message = 10003."article_u_id or u_id is null."
        response = dict(code=code, message=message, data="")
        return jsonify(response)
    with ClusterRpcProxy(CONFIG) as rpc:
        user_data = rpc.register.check_registered(u_id)
        if not user_data:
            code, message = 10004."You need to register to comment."
            response = dict(code=code, message=message, data="")
            return jsonify(response)

        # push message
        print("Push Message: article_u_id: {}".format(article_u_id))
        result, message = rpc.push.push(article_u_id, data.get("content"))
        print("push result: {}, message: {}".format(result, message))

    # save comment data
    print("Save Comment Data: article_id: {} content: {}".format(
        data.get("article_id"), data.get("content")))

    data = dict(comment_id=int(time.time()))
    response = dict(code=0, message="", data=data)
    return jsonify(response)
Copy the code

Comment interface, the description part is to provide the API interface description of Swagger (its specification needs to follow the Swagger specification, you can check the official document for details), and provide the user ID of the reviewer, the article ID, the content of the comment, and the user ID of the article author (for simplicity, directly provided by the client, The normal scenario is to find the author’s user ID based on the article ID).

The implementation of the function is also very simple, first through the call to check the registration service to see whether the commenter is registered, not directly return the need to register to comment. If already registered, the push service is invoked to send push notifications to the author. Then save the comment information and return the comment ID.

The key information is the realization of registration and push micro service, save comment information, I directly print here, without doing the actual operation.

@app.route('/api/v1/register', methods=['POST'])
def register(a):
    "" Register API Parameters Explain: timestamp Registered time email registered mailbox name Name Language Language country country -- Parameters: - name: body in: body required: true schema: id: data properties: timestamp: type: integer email: type: string name: type: string language: type: string country: type: string responses: code: description: 0 register success. message: description: Error Message! data: description: return u_id """

    user_data = request.json
    email = user_data.get("email")
    code, message = 0.""
    if not email:
        code, message = 10000."email is null."
        response = dict(code=code, message=message, data="")
        return jsonify(response)
    u_id = None
    with ClusterRpcProxy(CONFIG) as rpc:
        u_id, message = rpc.register.register(email, user_data)
    if message:
        code = 10001
    data = dict(u_id=u_id)
    response = dict(code=code, message=message, data=data)
    return jsonify(response)


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=int(parse_args.port), debug=True)
Copy the code

This is the last section of api.py, which implements the registration interface. Simply speaking, it calls the registration service. If it has been registered, it will return directly, otherwise it will store the user information.

The key is the implementation of the registration service.

Run python API. Py open http://localhost:5000/apidocs/ you can see the following interface:

Click on one of the apis to see the following interface:

Very convenient for API debugging.

Next comes the highlight, demonstrating the microservices code section:

import random from nameko.rpc import rpc import sys sys.path.append(".." ) from dependence.services import RedisService class RegisterService(object): name = "register" def __init__(self): self.redis_handle = RedisService() @rpc def check_registered(self, u_id): is_registered, user_data = self.redis_handle.check_registered_and_get_info(u_id) if is_registered: return user_data return None @staticmethod def generate_u_id(): """ Test Function """ return str(random.randint(7000000, 9999999)) @rpc def register(self, email, user_data): u_id = self.redis_handle.check_email_is_registered(email) if u_id: return u_id, "already registered." u_id = self.generate_u_id() register_result = self.redis_handle.register(u_id, email, user_data) if register_result: return u_id, "" return None, "register failed.Copy the code

Looking at the register implementation, you need to import nameko. RPC and decorate the function with RPC. The implementation is very simple. The code inside is the logical part, which generates the U_ID and stores it in Redis.

This implements the first function, calling the microservice in the API.

Let’s look at the implementation of the push service:

import random
from nameko.rpc import rpc, RpcProxy
import sys
sys.path.append("..")
from dependence.services import RedisService

class PushService(object):
    name = "push"
    register_rpc = RpcProxy("register")

    @rpc
    def push(self, u_id, content):
        user_data = self.register_rpc.check_registered(u_id)
        if not user_data:
        print("User:{} not existed.".format(u_id))
            return False."not registered."
        language, country = user_data["language"], user_data["country"]

        # get language push content
        print("Push Progress: u_id: {} language: {}, country: {}, content: {}".
              format(u_id, language, country, content))

        return True."push success."
Copy the code

The push service needs to call the registration service to determine whether the author of the article has registered (in fact, the author must have registered to publish the article, here refers to the demonstration), so that the micro-service is called in the micro-service, additional import RpcProxy is needed to specify the registration service RpcProxy(“register”). Then call the service, and get the user’s information, determine the language and country, push the corresponding language content.

Overall, Nameko framework, code layer implementation is very simple, lightweight, simple and practical. But it is not full of features, and there are not many Python back-end application scenarios.

3.2.3 commissioning

Open three terminals and run them respectively:cd microservices & nameko run push
cd microservices & nameko run register

cd app & python api.py
Copy the code

Open http://localhost:5000/apidocs/#/

Prepare registration data:

Registration Information 1 {"country": "CN"."email": "[email protected]"."language": "ZH"."name": "xiaohua"."timestamp": 1553652949} Registration information 2 {"country": "CN"."email": "[email protected]"."language": "ZH"."name": "xiaoming"."timestamp": 1553652950}Copy the code

Operation example:

Return message:

Return message 1 {"code": 0."data": {
    "u_id": "7434029"
  },
  "message": ""} Return message 2 {"code": 0."data": {
    "u_id": "8240184"
  },
  "message": ""
}
Copy the code

Call the push interface:

Click Execute.

Return message:

Python api.py:

Push Message: article_u_id: 7434029
push result: True, message: push success.
Save Comment Data: article_id: 100 content: very good.
127.0.0.1 - - [27/Mar/2019 23:24:07] "POST/API/v1 / comment HTTP / 1.1" 200 -
Copy the code

Github address github.com/CrystalSkyZ…

The next article will talk about RPC.

For more exciting articles, please pay attention to the public account “Tiancheng Technology Talk”

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

“Microservices & Nameko” Python Microservices practice

1. Microservices

2. Service Mesh and Serverless

2.1 Service Mesh

2.2 Serverless

3. Python framework for building microservices

3.1 Nameko Introduce

3.2 Practice

3.2.1 Environment Construction:

3.2.2 Code Demonstration:

3.2.3 commissioning

“Microservices & Nameko” Python Microservices practice

1. Microservices

2. Service Mesh and Serverless

2.1 Service Mesh

2.2 Serverless

3. Python framework for building microservices

3.1 Nameko Introduce

3.2 Practice

3.2.1 Environment Construction:

3.2.2 Code Demonstration:

3.2.3 commissioning

Related Posts

Speed reading “Message middleware” architecture -ActiveMQ: Introduction + Principle analysis + Advantages and Disadvantages!

Deploy k3S clusters across cloud vendors

JavaSE: chapter 16: new features in java8