preface
This article focuses on how to use Python to implement gRPC communication, with three examples going from simple to profound.
Note: this article is based on Python3
The project structure
The code structure of this paper is as follows. The source code is stored on my Github. If necessary, click here for reference.
School exercises ── Hello_client.py School exercises ─ hello_server.py School Exercises ─ helloWorld_PB2.py School Exercises ── HelloWorld_PB2_grpc.py School Exercises ── ├─ http_custom.py ├─ http_custom.py ├─ proto ├─ __init__Copy the codeDepend on the installation
1. Install gRPC
pip install grpcio
2. Install gRPC Tools
pip install grpcio-tools
Note: gRPC Tools includes the protobuf compiler, the Protoc, and the compiler plugin grpc_PYTHon_OUT (which will be used later for compilation).
A simple Demo
The simple Demo uses the official example and is relatively simple.
The interface definition
// file location: helloworld. Proto syntax = "proto3"; package helloworld; Service Greeter {// Basic Demo RPC SayHello (HelloRequest) returns (HelloResponse) {}} string name = 1; } // Simple response message HelloResponse {string message = 1; }Copy the codeA Greeter service provides a SayHello interface with a HelloRequest body that contains a name parameter. The response body is a HelloResponse that contains one parameter, Message.
Compile the proto
$ cd python_grpc
$ python -m grpc_tools.protoc -I./proto --python_out=. --grpc_python_out=. proto/helloworld.proto
Copy the codeAfter compilation, two stubs (helloWorld_pb2. py and helloWorld_PB2_grpc.py) are generated.
Server-side implementation
# hello_server.py
import grpc
import random
from concurrent import futures
import helloworld_pb2
import helloworld_pb2_grpc
# implement defined methods
class Greeter(helloworld_pb2_grpc.GreeterServicer) :
def SayHello(self, request, context) :
return helloworld_pb2.HelloResponse(message='Hello {msg}'.format(msg=request.name))
def serve() :
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
# bind handler
helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
server.add_insecure_port('[: :] : 50054')
server.start()
print('gRPC server enabled, port 50054... ')
server.wait_for_termination()
if __name__ == '__main__':
serve()
Copy the codeThe realization of the server side is mainly divided into two steps. The first step is to realize the interface we define, that is, to receive the request and return the response; The second step is to create a gRPC server instance, bind the handler and start it, listen for a particular port, and wait for the request to arrive.
Client-side implementation
# hello_client.py
import grpc
import helloworld_pb2, helloworld_pb2_grpc
def run() :
Channel is not secure because SSL is not used this time
channel = grpc.insecure_channel('localhost:50054')
# Client instance
stub = helloworld_pb2_grpc.GreeterStub(channel)
Call the server method
response = stub.SayHello(helloworld_pb2.HelloRequest(name='World'))
print("Greeter client received: " + response.message)
if __name__ == '__main__':
run()
Copy the codeAdvanced Demo
The above simple Demo is just for the convenience of telling the gRPC communication process, and the actual reference value is not too great. Because the field definition is too simple, a relatively complex example is provided here.
The interface definition
// file location: helloworld. Proto syntax = "proto3"; package helloworld; Service Greeter {// Advanced Demo RPC GetDeptUser (GetDeptUserRequest) returns (GetDeptUserResponse) {}} // Complex request message GetDeptUserRequest { uint32 dept_id = 1; // Department string dept_name = 2; // Department name Repeated uint32 UID_list = 3; // User ID list map<string, string> filter = 4; Message GetDeptUserResponse {repeated BasicUser user_list = 1; // User list MAP <uint32, BasicUser> user_map = 2; // user hash table} // BasicUser information message BasicUser {uint32 id = 1; // user hash table} // BasicUser information message BasicUser {uint32 id = 1; string name = 2; }Copy the codeNote: The interface compile commands are the same as above because they are defined in the same file
Server-side implementation
As for the server-side implementation, only the interface implementation section is listed, because the process of creating a gRPC server instance is the same as above.
Writing a:
class Greeter(helloworld_pb2_grpc.GreeterServicer) :
def GetDeptUser(self, request, context) :
The # field is obtained using a dot
dept_id = request.dept_id
dept_name = request.dept_name
uid_list = request.uid_list
if dept_id <= 0 or dept_name == ' ' or len(uid_list) <= 0:
return helloworld_pb2.GetDeptUserResponse()
print('dept_id is {0}, dept_name is {1}'.format(dept_id, dept_name))
user_list = []
user_map = {}
for id_ in uid_list:
uid = id_ + random.randint(0.1000)
letters = 'qwertyuiopasdfghjklzxcvbnm'
name = "".join(random.sample(letters, 10))
user = helloworld_pb2.BasicUser()
user.id = uid
user.name = name
user_list.append(user) This is similar to normal add operations
user_map[uid] = user
return helloworld_pb2.GetDeptUserResponse(user_list=user_list, user_map=user_map)
Copy the codeWrite two: define the object first, then assign the value
class Greeter(helloworld_pb2_grpc.GreeterServicer) :
def GetDeptUser(self, request, context) :
rsp = helloworld_pb2.GetDeptUserResponse()
dept_id = request.dept_id
dept_name = request.dept_name
uid_list = request.uid_list
if dept_id <= 0 or dept_name == ' ' or len(uid_list) <= 0:
return rsp
print('dept_id is {0}, dept_name is {1}'.format(dept_id, dept_name))
user_list = []
for id_ in uid_list:
uid = id_ + random.randint(0.1000)
letters = 'qwertyuiopasdfghjklzxcvbnm'
name = "".join(random.sample(letters, 10))
user = helloworld_pb2.BasicUser()
user.id = uid
user.name = name
user_list.append(user)
User_map [uid] = user rsp.user_map[uid] = user
rsp.user_map[uid].id = uid
rsp.user_map[uid].name = name
User_map = user_map, or rsp.user_map.update(user_map)
rsp.user_list.extend(user_list)
return rsp
Copy the codeClient-side implementation
Client is also written in two ways, similar to server.
Writing a:
response = stub.GetDeptUser(helloworld_pb2.GetDeptUserRequest(dept_id=1, dept_name='dd', uid_list=[1.2.3]))
print(response.user_list)
print(response.user_map)
Copy the codeMethod 2:
user_req = helloworld_pb2.GetDeptUserRequest()
user_req.dept_id = 110
user_req.dept_name = 'police'
user_req.uid_list.append(1)
user_req.uid_list.append(2)
user_req.uid_list.append(3)
The extend function can be used instead
# user_req.uid_list.extend([1, 2, 3])
response = stub.GetDeptUser(user_req)
print(response.user_list)
print(response.user_map)
Copy the codeUse HTTP as the mediator
Sometimes, the client and server do not directly communicate with each other through RPC, but do a layer conversion through HTTP. The common practice is to have an HTTP gateway, and the data from the client is serialized and transmitted to the server. The server parses the response data, and the client also parses the response data after receiving it.
Here’s another example to illustrate (the interface definition is the same as above) :
Server-side implementation
import random
import flask
from flask import request
import helloworld_pb2, helloworld_pb2_grpc
app = flask.Flask(__name__)
@app.route('/get_dept_user', methods=['POST'])
def get_dept_user() :
Receive client data and deserialize it
req_data = request.data
user_req = helloworld_pb2.GetDeptUserRequest()
user_req.ParseFromString(req_data)
# Splice response information
dept_id = user_req.dept_id
dept_name = user_req.dept_name
print('dept_id is {0}, dept_name is {1}'.format(dept_id, dept_name))
uid_list = user_req.uid_list
user_list = []
for id_ in uid_list:
uid = id_ + random.randint(0.1000)
letters = 'qwertyuiopasdfghjklzxcvbnm'
name = "".join(random.sample(letters, 10))
user = helloworld_pb2.BasicUser()
user.id = uid
user.name = name
user_list.append(user)
Serialize the response information
rsp = helloworld_pb2.GetDeptUserResponse()
rsp.user_list.extend(user_list)
rsp_data = rsp.SerializeToString()
return rsp_data
if __name__ == '__main__':
app.run('0.0.0.0', port=5001)
Copy the codeThe server side here uses Flask to build a simple HTTP server, where the client and server serialize data using SerializeToString and deserialize data using ParseFromString
Client-side implementation
import requests
import helloworld_pb2, helloworld_pb2_grpc
if __name__ == '__main__':
user_req = helloworld_pb2.GetDeptUserRequest()
user_req.dept_id = 110
user_req.dept_name = 'police'
user_req.uid_list.append(1)
user_req.uid_list.append(2)
user_req.uid_list.append(3)
If you do not use gRPC directly, but go through HTTP first, you need to serialize
data = user_req.SerializeToString()
req_url = 'http://127.0.0.1:5001/get_dept_user'
try:
response = requests.post(req_url, data=data)
ifresponse.status_code ! =200:
print('request failed, code: ', response.status_code)
Deserialize data
rsp = helloworld_pb2.GetDeptUserResponse()
rsp.ParseFromString(response.content)
print('rsp: ', rsp.user_list)
except Exception as e:
print('request failed, err: ', e)
Copy the codeThe client here uses the Request framework to send requests, and there are also serialization and deserialization processes.
Write in the last
If you find gRPC useful, please give me a thumbs up!
I am Yan Gan, welcome to follow my public account Yan Gan Coding, your likes and attention are the biggest motivation for my creation! !
Note: remind again, the source code at the top of the address oh!
reference
Python gRPC Quick Start Python gRPC Quick Start