GRPC: How to enable GRPC to provide Restful API services?

introduce

This article will show you how to make a gRPC service that provides both gRPC and Restful apis.

To enable gRPC to provide REST APIS, we need to use grPC-Gateway

Please visit the following address for the full tutorial:

rkdev.info/cn

Rkdocs.net lilify. app/cn (standby)

Use fairly rk – the boot

Rk-boot is a launcher that integrates Gin, gRPC and a series of popular Go language frameworks. Users can quickly start enterprise-class Go language microservices through RK-Boot.

A prerequisite for

As anyone who has used GRPC should know, the protocol buffer requires compiling the *. Proto file into the *. Go file using the associated command line.

Depending on your needs, different command line files are used. Take the Go language as an example, we need the following command line files.

tool	introduce	The installation
protobuf	Protocol buffer Command line required for compilation	Install
protoc-gen-go	From the proto file, generate the.go file	Install
protoc-gen-go-grpc	From the proto file, generate the grPC-related.go file	Install
protoc-gen-grpc-gateway	Generate the. Go file related to grPC-gateway from the proto file	Install
protoc-gen-openapiv2	From the proto file, generate the parameter files required by the Swagger interface	Install

In addition to installing the above command line, we need to run at least four different commands to compile the *. Proto file as needed, which is quite obscure.

For details, please refer to my previous article: [GRPC: Using Buf to quickly compile GRPC proto files] or visit: [rkdev.info/cn/docs/boo…

The installation

go get github.com/rookie-ninja/rk-boot
Copy the code

Quick start

1. Create API/v1 / greeter. Proto

syntax = "proto3";

package api.v1;

option go_package = "api/v1/greeter";

service Greeter {
  rpc Greeter (GreeterRequest) returns (GreeterResponse) {}
}

message GreeterRequest {
  string name = 1;
}

message GreeterResponse {
  string message = 1;
}
Copy the code

2. Create the API/v1 / gw_mapping. Yaml

type: google.api.Service
config_version: 3

# Please refer google.api.Http in https://github.com/googleapis/googleapis/blob/master/google/api/http.proto file for details.
http:
  rules:
    - selector: api.v1.Greeter.Greeter
      get: /api/v1/greeter
Copy the code

3. Create buf. Yaml

version: v1beta1
name: github.com/rk-dev/rk-demo
build:
  roots:
    - api
Copy the code

4. Create buf. Gen. Yaml

version: v1beta1
plugins:
  # protoc-gen-go needs to be installed, generate go files based on proto files
  - name: go
    out: api/gen
    opt:
     - paths=source_relative
  # protoc-gen-go-grpc needs to be installed, generate grpc go files based on proto files
  - name: go-grpc
    out: api/gen
    opt:
      - paths=source_relative
      - require_unimplemented_servers=false
  # protoc-gen-grpc-gateway needs to be installed, generate grpc-gateway go files based on proto files
  - name: grpc-gateway
    out: api/gen
    opt:
      - paths=source_relative
      - grpc_api_configuration=api/v1/gw_mapping.yaml
  # protoc-gen-openapiv2 needs to be installed, generate swagger config files based on proto files
  - name: openapiv2
    out: api/gen
    opt:
      - grpc_api_configuration=api/v1/gw_mapping.yaml
Copy the code

5. Compile proto file

$ buf generate
Copy the code

The following files will be created.

$tree API/API/gen gen └ ─ ─ v1 ├ ─ ─ greeter. Pb. Go ├ ─ ─ greeter. Pb. Gw go ├ ─ ─ greeter. Swagger. Json └ ─ ─ greeter_grpc. Pb. Go 1 directory, 4 filesCopy the code

6. Create the boot. Yaml

grpc:
  - name: greeter                   # Name of grpc entry
    port: 8080                      # Port of grpc entry
    enabled: true                   # Enable grpc entry
    enableReflection: true
Copy the code

7. Create a main. Go

package main import ( "context" "fmt" "github.com/rookie-ninja/rk-boot" "github.com/rookie-ninja/rk-demo/api/gen/v1" "google.golang.org/grpc" ) // Application entrance. func main() { // Create a new boot instance. boot := rkboot.NewBoot() // *************************************** // ******* Register GRPC & Gateway ******* // *************************************** // Get grpc entry with name grpcEntry := boot.GetGrpcEntry("greeter") // Register grpc registration function grpcEntry.AddRegFuncGrpc(registerGreeter) // Register grpc-gateway registration function grpcEntry.AddRegFuncGw(greeter.RegisterGreeterHandlerFromEndpoint) // Bootstrap boot.Bootstrap(context.Background()) // Wait for shutdown sig boot.WaitForShutdownSig(context.Background()) } // Implementation of [type GrpcRegFunc func(server *grpc.Server)] func registerGreeter(server *grpc.Server) { greeter.RegisterGreeterServer(server, &GreeterServer{}) } // Implementation of grpc service defined in proto file type GreeterServer struct{} func (server *GreeterServer) Greeter(ctx context.Context, request *greeter.GreeterRequest) (*greeter.GreeterResponse, error) { return &greeter.GreeterResponse{ Message: fmt.Sprintf("Hello %s!" , request.Name), }, nil }Copy the code

8. Folder structure

$tree. ├ ─ ─ API │ ├ ─ ─ gen │ │ └ ─ ─ v1 │ │ ├ ─ ─ greeter. Pb. Go │ │ ├ ─ ─ greeter. Pb. Gw go │ │ ├ ─ ─ greeter. Swagger. Json │ │ ├── ├─ trash ├── ├─ trash ├── trash ├── trash ├── trash ├── trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash ├─ trash └── Go. ├─ go. ├─ main. Go directories, 12 filesCopy the code

9. Verify

Rk-boot uses the same port to map gRPC and GRPC-gateway, which is very convenient.

$ go run main.go
Copy the code

Verify the Restful API

$ curl "localhost:8080/api/v1/greeter? name=rk-dev" {"message":"Hello rk-dev!" }Copy the code

Verify the gRPC

$ grpcurl -d '{"name": "rk-dev"}' -plaintext localhost:8080 api.v1.Greeter.Greeter
{
  "message": "Hello rk-dev!"
}
Copy the code

Gateway server option

Rk-boot provides the enableRkGwOption option. After rK-boot is enabled, it provides the following functions.

The server option recommended by RK provides the following functions.

rkgrpc.RkGwServerMuxOptions

function	details
HttpErrorHandler	The main code is copied from the original GRPC-gateway code, and the initiator returns the API error structure recommended by RK
MarshalerOption	protojson.MarshalOptions{UseProtoNames: false, EmitUnpopulated: true},UnmarshalOptions: protojson.UnmarshalOptions{}}
Metadata	Inject GRPC metadata as follows: X-forwarded-method, X-Forwarded-Path, X-Forwarded-Scheme, X-Forwarded-User-Agent, and X-Forwarded-remote-addr
OutgoingHeaderMatcher	Forwarding the original GRPC metadata to the HTTP header (without prefix)
IncomingHeaderMatcher	Forwarding the original HTTP header to GRPC Metadata (without prefix)

1. Start the enableRkGwOption

To verify, we will add a gRPC log blocker.

boot.yaml

grpc:
  - name: greeter                   # Name of grpc entry
    port: 8080                      # Port of grpc entry
    enabled: true                   # Enable grpc entry
    enableReflection: true          # Enable grpc reflection
    enableRkGwOption: true          # Enable rk style grpc gateway server option
    interceptors:
      loggingZap:
        enabled: true               # Enable logging interceptor for validation
Copy the code

2. Verify logs

$ curl "localhost:8080/api/v1/greeter? name=rk-dev"Copy the code

GwMethod, gwPath, gwScheme, and gwUserAgent will be logged

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- endTime = 2021-07-09 T21:03:43. 518106 + 08:00... payloads={"grpcMethod":"Greeter","grpcService":"api.v1.Greeter","grpcType":"unaryServer","gwMethod":"GET","gwPath":"/v1/ Greeter gwScheme ", "" :" HTTP ", "gwUserAgent" : "curl / 7.64.1"}...Copy the code

3. Verify the incoming metadata

func (server *GreeterServer) Greeter(ctx context.Context, request *greeter.GreeterRequest) (*greeter.GreeterResponse, error) { // Print incoming headers fmt.Println(rkgrpcctx.GetIncomingHeaders(ctx)) return &greeter.GreeterResponse{ Message: fmt.Sprintf("Hello %s!" , request.Name), }, nil }Copy the code

The map [: authority: [0.0.0.0:8080] the accept: * / * the content-type: application/GRPC the user-agent: [GRPC - go / 1.38.0] x-forwarded-for:[::1] x-forwarded-host:[localhost:8080] x-forwarded-method:[GET] x-forwarded-path:[/v1/greeter] X - forwarded - remote - addr: [[: : 1) : 57082] x - forwarded - scheme: [HTTP] x - forwarded - the user-agent: [curl / 7.64.1]]Copy the code

Error mapping

In grPC-gateway, we need to understand the incorrect mapping of gateway to GRPC, that is, the incorrect mapping of HTTP to GRPC.

This is the default error mapping in GRPC-gateway.

GRPC error code	GRPC Error code description	Gateway (Http) error code	Gateway(Http) error code description
0	OK	200	OK
1	CANCELLED	408	Request Timeout
2	UNKNOWN	500	Internal Server Error
3	INVALID_ARGUMENT	400	Bad Request
4	DEADLINE_EXCEEDED	504	Gateway Timeout
5	NOT_FOUND	404	Not Found
6	ALREADY_EXISTS	409	Conflict
7	PERMISSION_DENIED	403	Forbidden
8	RESOURCE_EXHAUSTED	429	Too Many Requests
9	FAILED_PRECONDITION	400	Bad Request
10	ABORTED	409	Conflict
11	OUT_OF_RANGE	400	Bad Request
12	UNIMPLEMENTED	501	Not Implemented
13	INTERNAL	500	Internal Server Error
14	UNAVAILABLE	503	Service Unavailable
15	DATA_LOSS	500	Internal Server Error
16	UNAUTHENTICATED	401	Unauthorized

1. Validation error (standard Go language error)

According to the error mapping, 500 will be returned.

func (server *GreeterServer) Greeter(ctx context.Context, request *greeter.GreeterRequest) (*greeter.GreeterResponse, error) {
   return nil, errors.New("error triggered manually")
}
Copy the code

$ curl "localhost:8080/api/v1/greeter? name=rk-dev" { "error":{ "code":500, "status":"Internal Server Error", "message":"error triggered manually", "details":[] } }Copy the code

2. Validation error (GRPC error)

We need to create GRPC errors with status.new (). We recommend using functions in the Rkerror library to create errors.

func (server *GreeterServer) Greeter(ctx context.Context, request *greeter.GreeterRequest) (*greeter.GreeterResponse, error) {
   return nil, rkerror.PermissionDenied("permission denied manually").Err()
}
Copy the code

curl "localhost:8080/v1/greeter? name=rk-dev" { "error":{ "code":403, "status":"Forbidden", "message":"permission denied manually", "details":[ { "code":7, "status":"PermissionDenied", "message":"[from-grpc] permission denied manually" } ] } }Copy the code