Small knowledge, big challenge! This article is part of the “Programmer’s Essentials

This article also participated in the “Digitalstar Project” to win a creative gift package and creative incentive money

I’m going to start working on a gameserver project, so I’m going to start working on using Protobuf for the network protocol, which is the most popular and most used one right now, so I’m going to write protobuf today, even though it’s been used throughout the project, but it’s just inertia, so I’m just going to review it. Avoid jamming during use. Let’s go.

Note: I wrote protobuf in Java based on my experience with Java because I am not familiar with other languages and have not had much experience, but it is similar.

1. Build protobuf environment

Protobuf is a set of methods developed by Google for serializing data structures that can be used as communication protocols, data storage formats, etc. It is characterized by unlimited language, unlimited platform, and strong extensibility, just like XML. Compared to XML, Protobuf has the following features:

1.1 Protobuf version

Address of the latest version: Dependency · Protocolbuffers /protobuf · GitHub

Maven relies on configuration, so just put it in there without thinking too much

< the dependency > < groupId > com. Google. Protobuf < / groupId > < artifactId > protobuf - Java < / artifactId > < version > 3.18.0 < / version > </dependency>Copy the code

1.2 Proto compiler

The home page has been pulled down, can see protoc various system versions, according to their own operating system selection, I am a 64-bit Win10, so the choice of Win64, download decompression is good, such as use

2. Protobuf syntax

Proto3 syntax document: developers.google.com/protocol-bu…

syntax = "proto3";
​
message SearchRequest {
  string query = 1;
  int32 page_number = 2;
  int32 result_per_page = 3;
}
Copy the code

2.1 Comment Rules

Add comments to your.proto using C/C++ style // or /*… * / syntax.

/* SearchRequest represents a search query, with pagination options to * indicate which results to include in the response. */
​
message SearchRequest {
  string query = 1;
  int32 page_number = 2;  // Which page number do we want?
  int32 result_per_page = 3;  // Number of results to return per page.
}
Copy the code

2.2 Data Types

It’s basically just regular data types

Values: int32, int64, float,double, s, u, fix(fixed length, not used)

Boolean value: bool

String: string

Enumeration: enum

List: repeated

Binary: bytes

So basically what you use is these simple types, nothing special.

2.3 Default value rules Define the default value of data types. What are the default value rules when data types are not transmitted

String: The default value is”

Bool: The default value is false

Value type: The default value is 0, 0.f or 0.d.

Enumeration: The default value is the first enumeration value, which is 0

Message compound type: null in Java.


message SearchResponse {
  repeated Result results = 1;
}
​
message Result {
  string url = 1;
  string title = 2;
  repeated string snippets = 3;
}
Copy the code

2.4 Protobuf Options Protobuf provides some options for setting generated Proto files

option java_package = "com.example.foo";
option java_outer_classname = "Ponycopter";
option java_multiple_files = true;
option optimize_for = CODE_SIZE;
Copy the code

option java_package = “com.example.foo”; The package name of the generated Java file

option java_outer_classname = “Ponycopter”; The name of the generated class

option java_multiple_files = true; False generates the entire proto file in a Java file. True generates the entire Proto file in a Java file

option optimize_for = CODE_SIZE; You can set it to SPEED, CODE_SIZE, LITE_RUNTIM

SPEED: Priority is given to SPEED of code sequences and conversions

CODE_SIZE: Optimized generated code size

LITE_RUNTIME: Generated code requires less runtime. This option is generally not used.

Copy the protoc.exe you downloaded to the location of your proto file, right click in the folder shfit + and select open CMD window to run the following


protoc --java_out=./ XX.proto
Copy the code

Java_out is the generated output address

Xx. proto is the proto file you want to compile

Try it

3. Idea generation plug-in

Java development is the most commonly used is IDEA, because IDEA powerful plug-in system is really easy to use, protobuf development in IDEA is also very convenient, here recommend two proto plug-in, help you in the development of the tiger with wings.

3.1 GenProtobuf

Generating proto every time input command is a bit annoying, so someone wrote a plugin genProtobuf, in the idea dot can do,

Install: Simple, File -> Settings->Plugins, click Install, and wait until the installation is complete

Configuration: Tools -> Config GenProtobuf

Generate Java files: Select the Proto file to be generated and quick Gen Protobuf Rules to generate Java files based on the rules configured in the previous step

3.2 Protobuf The protobuf plugin supports proto file syntax and highlights keywords

Serialization and deserialization

4.1 General serialization and deserialization

Routine use


        SimpleMessage.Builder builder = SimpleMessage.newBuilder();
        builder.setName("Parsley");
        builder.setId(1);
        builder.setIsSimple(true);
        byte[] result=builder.build().toByteArray();/ / the serialization
​
        SimpleMessage msg;
        try {
            msg = SimpleMessage.parseFrom(result);
            System.out.println(msg);
        } catch (InvalidProtocolBufferException e) {
            e.printStackTrace();
        }
Copy the code

4.2 Universal deserialization

Use parser to deserialize and save the parser for each message. The message can be scanned when started and the protocol corresponding to the message ID can be saved

import com.google.protobuf.Parser;
import com.xin.msg.login.SimpleMessage;
​
import java.util.HashMap;
import java.util.Map;
​
public class MsgMgr {
    public static Map<Integer, Parser> msgMap = new HashMap<>();
    static {
        // msgId - parser
        msgMap.put(1, SimpleMessage.parser()); }}Copy the code

In the process of parsing, the corresponding parser is obtained according to the message Id and converted into the corresponding message


        Parser parser = MsgMgr.msgMap.get(headId);
        int canReadBytes = decode.readableBytes();
        byte[] data = new byte[canReadBytes];
        decode.readBytes(data);
        Object o = parser.parseFrom(data);
Copy the code

With protobuf, you need to write proto file, this file is used to synchronize with the client, actually a bit annoying, and then compile it into the target language, so it is very troublesome to use. But now with Protostuff, there is no need to rely on.proto files, it can serialize and deserialize POJOs directly, it is very easy to use.

Github – protostuff/protostuff: Java Serialization Library, Proto Compiler, Code Generator github.com/protostuff/…

Maven dependency, add to your pom.xml

<dependency>
  <groupId>io.protostuff</groupId>
  <artifactId>protostuff-core</artifactId>
  <version>1.7.4</version>
</dependency>
<dependency>
  <groupId>io.protostuff</groupId>
  <artifactId>protostuff-runtime</artifactId>
  <version>1.7.4</version>
</dependency>
Copy the code

Instance;

public final class Foo
{
    String name;
    int id;
    
    public Foo(String name, int id)
    {
        this.name = name;
        this.id = id; }}static void roundTrip(a)
{
    Foo foo = new Foo("foo".1);
​
    // this is lazily created and cached by RuntimeSchema
    // so its safe to call RuntimeSchema.getSchema(Foo.class) over and over
    // The getSchema method is also thread-safe
    Schema<Foo> schema = RuntimeSchema.getSchema(Foo.class);
​
    // Re-use (manage) this buffer to avoid allocating on every serialization
    LinkedBuffer buffer = LinkedBuffer.allocate(512);
​
    // ser
    final byte[] protostuff;
    try
    {
        protostuff = ProtostuffIOUtil.toByteArray(foo, schema, buffer);
    }
    finally
    {
        buffer.clear();
    }
​
    // deser
    Foo fooParsed = schema.newMessage();
    ProtostuffIOUtil.mergeFrom(protostuff, fooParsed, schema);
}
​
Copy the code

Protostuff will not be specifically introduced, and the official address has been attached. Meanwhile, I did not use it in the project. The reason is that THE PROTO file needs to be sent to the client for short use, and proTO is a bridge for protocol communication with the client.

6, summary

Protobuf is just a communication protocol, it has a lot of details, but it doesn’t need to be too deep, it’s not too late for you to check the documentation when you encounter problems, remember the common data types, it doesn’t affect moving bricks in the work, come on, use it next.