Kratos uses -protobuf

What is the ProtoBuf

Protocol Buffers is a portable and efficient structured data storage format that can be used for structured data serialization and is suitable for data storage or RPC data exchange. It can be used for language – independent, platform – independent and extensible serialized structure data format in communication protocol, data storage and other fields

ProtoBuf is specifically used to describe the data format of interactions between clients and services, as well as the parsing and serialization of messages, and has the following advantages:

Both transfer efficiency and parsing efficiency are much higher than text serialization formats (e.g., JSON, XML, etc.). Platform-independent, language-independent, extensible serialization structure data formats
With a uniform syntax for describing interactive data structures, tools can easily automate the generation of API documents (e.g., Swagger documents)
Mandatory proto file + automatic document generation can largely solve the problem of document update before the client and service
Google does the endorsement, the ecosystem is perfect, each language library has support

Syntactic shorthand

// Version definition, the most recent uses proto3
syntax = "proto3";

// Common usage
import "google/protobuf/any.proto";

// Import a third party proto file through public, then import your file will also import a third party proto
import public "other2.proto";

// Define the package name of proto to avoid name conflicts between types of message
package foo.bar;

// Option is defined as "option" optionName "=" constant ";"
option java_package = "com.example.foo";

message OneofMessage {
  // Reserved Fields can be specified by field number range or field name. These fields will be ignored
  reserved 2.4 to 6;
  reserved "field14"."field11";

  double field1 = 1;
  // float field2 = 2;
  int32 field3 = 3;
  // int64 field4 = 4;
  // uint32 field5 = 5;
  // uint64 field6 = 6;
  sint32 field7 = 7;
  sint64 field8 = 8;
  fixed32 field9 = 9;
  fixed64 field10 = 10;
  // sfixed32 field11 = 11;
  sfixed64 field12 = 12;
  bool field13 = 13;
  // string field14 = 14;
  bytes field15 = 15;

      // a group of fields, at most one of which is allowed at the same time
  oneof test_oneof {
    string name = 16;
    int64 value = 17;
  }

  // Map fields cannot be repeated at the same time
  map<int64.string> values = 18;

  // Messages can be directly embedded with enumerations
  enum Corpus {
    UNIVERSAL = 0;
    WEB = 1;
    IMAGES = 2;
  }
  Corpus corpus = 19;
  	
  // repeated is a list, which can be repeated to other types
  repeated Result results = 1;
}

message Result {
  string url = 1;
  string title = 2;
  repeated string snippets = 3;
}

// The enumeration constant must be a 32 bit integer. Negative numbers are not recommended for efficiency reasons
// The first enumeration value must be 0 and must be defined
enum EnumAllowingAlias {
	Allow_alias: allow_alias: allow_alias: allow_alias: allow_alias
  option allow_alias = true;
  UNKNOWN = 0;
  STARTED = 1;
  RUNNING = 1;
}
Copy the code

Note for version iteration

Do not change the field number of an existing field
When you add a new field, the serialized data of the old system can still be parsed by your new format, but you need to deal with the default values of the new field. The old system can also parse the value of your message, the new fields are just discarded
Fields can also be removed, but it is recommended that you reserve this field to avoid using it in the future
Int32, uint32, INT64, uint64 Compatible with bool
Sint32 is compatible with sint64, but not with other integer types
String is compatible with bytes, if bytes is a valid UTF-8 bytes
The embedded type is compatible with bytes if bytes contains an encoded version of the message
Fixed32 and sFixed32, fixed64 and sfixed64
Enum and INT32, uint32, INT64, uint64 Compatible format
Changing a single value to a member of a new oneof type is safe and binary compatible. Turning a set of fields into a new oneof field is also safe if you ensure that the set of fields will only be set to one at most. It is not safe to move a field to an existing oneof field

Applications in Kratos

After a new project is created using kratos scaffolding, the proto files are organized in the API directory under the root directory. We create a v1 directory under the API to hold all the interface definitions of the first version. The directory structure is as follows:

├ ─ garbage ─ 1.1.55Copy the code

Then write the user.proto file

syntax = "proto3";

import "github.com/gogo/protobuf/gogoproto/gogo.proto";
import "google/protobuf/empty.proto";

package user.v1;

option go_package = "v1";
option (gogoproto.goproto_getters_all) = false;

service user {

  rpc mine_info(GetMineInfoReq) returns (GetMineInfoResp);
}

message GetMineInfoReq {
  // User ID
  int64 uid = 1 [(gogoproto.moretags) = 'form:"uid" validate:"required"'];
}

message GetMineInfoResp {
  // The user's account
  string account = 1 [(gogoproto.jsontag) = 'account'];
  // Check whether the small B account has been authenticated
  bool auth = 2 [(gogoproto.jsontag) = 'auth'];
  // User profile picture
  string avatar = 3 [(gogoproto.jsontag) = 'avatar'];
  / / user name
  string name = 4 [(gogoproto.jsontag) = 'name'];
}
Copy the code

After writing, run the kratos tool protoc user.proto command in the v1 directory to generate all types of files by default, for example: API. Pb. Go/API. Bm. Go/API. The swagger. Json/API. Ecode. Go the corresponding files

But at the time of execution error, says there are several libraries is not defined, these are some of the third-party libraries, in the first line of the printing can see protoc command execution of the original command, you can see the introduction of several libraries folders do not Google/protobuf/descriptor. The proto this library

[16:50:26] jeff:v1 $ kratos tool protoc user.proto 2020/11/19 16:50:35 protoc --proto_path=/Users/jeff/go/src - proto_path=/Users/jeff/go/pkg/mod/github.com/go-kratos/[email protected]/third_party --proto_path=/Users/jeff/Desktop/workspace/others/kratos-test/user-service/api/v1 --bm_out=:. user.proto google/protobuf/descriptor.proto: File not found. github.com/gogo/protobuf/gogoproto/gogo.proto:32:1: Import "google/protobuf/descriptor.proto" was not found or had errors. github.com/gogo/protobuf/gogoproto/gogo.proto:38:8: "google.protobuf.EnumOptions" is not defined. github.com/gogo/protobuf/gogoproto/gogo.proto: "google.protobuf.EnumOptions" is not defined. ... Ignore here... github.com/gogo/protobuf/gogoproto/gogo.proto: "google.protobuf.FieldOptions" is not defined. google/protobuf/empty.proto: File not found. user.proto:3:1: Import "github.com/gogo/protobuf/gogoproto/gogo.proto" was not found or had errors. user.proto:4:1: Import "google/protobuf/empty.proto" was not found or had errors. exit status 1 exit status 1Copy the code

Protobuf libraries are installed in /usr/local/include/directory

[17:20:14] jeff:v1 $ ll /usr/local/include | grep google lrwxr-xr-x 1 jeff admin 39B Sep 9 2019 google -> .. / Cellar/protobuf / 3.7.1 / include/Google [17:20:14] Jeff: v1 $[17:20:45] Jeff: v1 $kratos tool protoc user. The proto --proto_path=/usr/local/include/ 2020/11/19 17:20:47 protoc --proto_path=/Users/jeff/go/src - proto_path=/Users/jeff/go/pkg/mod/github.com/go-kratos/[email protected]/third_party --proto_path=/Users/jeff/Desktop/workspace/others/kratos-test/user-service/api/v1 --bm_out=:. user.proto --proto_path=/usr/local/include/ user.proto:4:1: warning: Import google/protobuf/empty.proto is unused. 2020/11/19 17:20:47 protoc --proto_path=/Users/jeff/go/src - proto_path=/Users/jeff/go/pkg/mod/github.com/go-kratos/[email protected]/third_party --proto_path=/Users/jeff/Desktop/workspace/others/kratos-test/user-service/api/v1 --gofast_out=plugins=grpc:. user.proto  --proto_path=/usr/local/include/ user.proto:4:1: warning: Import google/protobuf/empty.proto is unused. 2020/11/19 17:20:47 protoc --proto_path=/Users/jeff/go/src - proto_path=/Users/jeff/go/pkg/mod/github.com/go-kratos/[email protected]/third_party --proto_path=/Users/jeff/Desktop/workspace/others/kratos-test/user-service/api/v1 --bswagger_out=:. user.proto --proto_path=/usr/local/include/ user.proto:4:1: warning: Import google/protobuf/empty.proto is unused. 2020/11/19 17:20:47 protoc --proto_path=/Users/jeff/go/src - proto_path=/Users/jeff/go/pkg/mod/github.com/go-kratos/[email protected]/third_party --proto_path=/Users/jeff/Desktop/workspace/others/kratos-test/user-service/api/v1 --ecode_out=:. user.proto --proto_path=/usr/local/include/ user.proto:4:1: warning: Import google/protobuf/empty.proto is unused. 2020/11/19 17:20:47 generate user.proto --proto_path=/usr/local/include/ success.Copy the code

gogoprotobuf

Another option for using protobuf in GO is GogoProtobuf. In addition to being compatible with Protobuf, GogoProtobuf adds many options, so it provides lower granularity control, more efficient and reasonable code generation. Etcd, K8S, Dgraph, Docker SwarmKit all use it and Kratos uses the same library

Gogoprotobuf adds a number of options at different levels, see GogoProto Extensions for details

option name	option	describe
gogoproto.goproto_enum_prefix	Enum	When the option is false, the generated code does not include “E_”
gogoproto.goproto_getters	Message	When the option is false, a Get function is not generated for each field of message
gogoproto.face	Message	When the option is true, the corresponding interface is generated for Message
gogoproto.nullable	Field	When the option is false, goGo sets a value for each field of message when the message is serialized
gogoproto.goproto_stringer	Message	When the option is false, gogo no longer generates String() methods for message on a struct

Goland introduces an issue where third party libraries cannot be found when using Protobuf Support

– Goland install Protobuf Support and introduce libraries such as Gogo. Error message “File not found”, it does not affect use, it is not very beautiful when writing, it also loses static check for spelling error of option

This problem mainly because of the plugin repository is not we need third-party libraries, so can’t find, and Google/protobuf/empty. The proto this library is not an error because the plug-in integrates the library inside, according to the command button you can see, The Library is the address of the in/Users/Jeff/Library/Application Support/JetBrains/GoLand2020.2 / plugins/protobuf – JetBrains – the plugin/lib/protobuf – Java – 3.6.1 track – sources. The jar directory

Unzip and jar command can complete jar package unpack and package, put gogoProtobuf library related to it, and then replace the previous library JAR package can be

The error disappears directly after replacement and can be traced back to the library source by using the Command key

What is the ProtoBuf

Syntactic shorthand

Note for version iteration

Applications in Kratos

gogoprotobuf

Goland introduces an issue where third party libraries cannot be found when using Protobuf Support

Related Posts

The Book (4) the Elasticsearch in actual combat

Distributed message flow platforms: Don’t just think Kafka, but Pulsar

(Design pattern) Observer