preface

In the last article we talked about codecs and using Java serialization as codecs as a parsing framework. Originally intended as a long document to cover codec and its application. But find it more clear to write separately in real time, just as a record.

Google Protobuf

What is a Protobuf? I took a line from the Google Developer Center

Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data.

Protobuf is a cross-language/cross-platform, extendable, serializable data structure. Protobuf is Google’s language-independent, platform-independent, extensible mechanism for serializing structured data.

It describes data structures as.proto files, and code generation tools can generate POJO objects and protobuf-related methods and properties for the corresponding data structures. You only need to define how your data is structured once, and then you can easily write and read structured data into a variety of data streams, using specially generated source code, in multiple languages.

So what are the supported languages? It supports C++ / C# / Dart/Go/Java as well as Python. It can be said that most of the major languages currently support it. The cross-language benefit, as I mentioned, is the ability to seamlessly switch data exchange between heterogeneous language systems, an advantage that Java serialization cannot match. The exception is that Protobuf has other advantages, for example

  1. The serialized version is smaller because it is binary and better suited for network transport
  2. Message format upgrade compatibility is good
  3. Serialization and deserialization are fast

So, now let’s do a simple demo using Netty and Protobuf.

Book inquiry system

Our next demo is a simple book query system, the business process is as follows

The client sends the name of the query to the server, and the server returns basic information about the book query

Construct object

Using a Protobuf to construct an object has the following steps:

  1. Download for compilationproto.exefile
  2. Writes information about the object to be constructedprotobuffile
  3. throughproto.exeTo generate aJava POJOobject
  4. usePOJOConduct application development

downloadProtobuf

Download the Protobuf we can go to dead simple to download under https://github.com/protocolbuffers/protobuf/releases.

There are OS system, Windows system, and Linux system support, so you can choose the corresponding version according to the system.

I have chosen the Windows system for development, so my version is Protoc-3.12.3-win64.zip.

Create the.proto file for the object

We need to write the information about the object to be constructed to a Protobuf file. The first thing we construct is the SubscribeReq request class.

SubscribeReq.proto

package netty;

option java_package="codec.protobuf";
option java_outer_classname="SubscribeReqProto";

message  SubscribeReq{
    required int32 subReqID = 1;
    required string userName = 2;
    required string productName = 3;
    required string address = 4;
}

Copy the code

Let me explain, package means the folder that was generated; Java_package refers to the generated POJO object that is currently under your project package; Java_outer_classname refers to the actual classname of the generated POJO object.

The client requests the server request entity class constructed above, followed by the reply class constructed for the server to answer the client.

SubscribeResp.proto

package netty;

option java_package="codec.protobuf";
option java_outer_classname="SubscribeRespProto";

message  SubscribeResp{
    required int32 subReqID = 1;
    required int32 respCode = 2;
    required string desc = 3;
}
Copy the code

After the construction, we continue to go down!

useproto.exeTo generate the object

The generation step is actually to generate POJO objects from the specified.proto file using proto.exe from the command line (Window CMD).

Since we are in the window integration environment, we need to open CMD. Then use the command to go to the protobuf folder you just downloaded. Generate the POJO object with the following command

protoc.exe -I=$SRC_DIR --java_out=$DST_DIR +$SRC_DIR/target.proto
Copy the code

Where javA_out is the path to Java output, followed by $SRC_DIR/target.proto, which represents proTO about the target.

So now we compile the two files we use. For convenience, I will copy both files to the same directory as proto.exe. Open CMD and go to the relative directory of proto.exe.

protoc.exe  --java_out=./ ./SubscribeReq.proto
Copy the code

For a brief explanation, compile the SubscribeReq. Proto under the current path and output the generated results to the current folder. This is followed by the compilation of the reply file

protoc.exe  --java_out=./ ./SubscribeResp.proto
Copy the code

Next you can see if the corresponding file is generated in the same directory as./proto.exe.

Use POJOs for application development

Now the formal business development begins. First we need to copy the generated POJOs to the project directory.

Environment depends on
    <dependencies>

<! -- netty -->
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-all</artifactId>
            <version>4.1.50. The Final</version>
        </dependency>

<! -- protobuf -->
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>3.12.2</version>
        </dependency>
    </dependencies>
Copy the code
The service side

The first is the server startup class.

public class SubReqServer {
    public void bind(int port) throws Exception {
        // Configure the NIO thread group on the server
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 100)
                .handler(new LoggingHandler(LogLevel.INFO))
                .childHandler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    public void initChannel(SocketChannel ch) {
                     ch.pipeline().addLast(
                     new ProtobufVarint32FrameDecoder());
                    ch.pipeline().addLast(
                            new ProtobufDecoder( SubscribeReqProto.SubscribeReq.getDefaultInstance()));
                    ch.pipeline().addLast(
                            new ProtobufVarint32LengthFieldPrepender());
                    ch.pipeline().addLast(new ProtobufEncoder());
                    ch.pipeline().addLast(newSubReqServerHandler()); }});// Bind the port and wait for synchronization successfully
            ChannelFuture f = b.bind(port).sync();
            // Wait for the server listening port to close
            f.channel().closeFuture().sync();
        } finally {
            // Exit gracefully to free up thread pool resourcesbossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); }}public static void main(String[] args) throws Exception {
        new SubReqServer().bind(8080); }}Copy the code

Then there are the processing classes for the server

@Sharable
public class SubReqServerHandler extends ChannelInboundHandlerAdapter {
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg)
            throws Exception {
        SubscribeReqProto.SubscribeReq req = (SubscribeReqProto.SubscribeReq) msg;
        if ("netty".equalsIgnoreCase(req.getUserName())) {
            System.out.println("Service accept client subscribe req : ["
                    + req.toString() + "]"); ctx.writeAndFlush(resp(req.getSubReqID())); }}private SubscribeRespProto.SubscribeResp resp(int subReqID) {
        SubscribeRespProto.SubscribeResp.Builder builder = SubscribeRespProto.SubscribeResp
                .newBuilder();
        builder.setSubReqID(subReqID);
        builder.setRespCode(0);
        builder.setDesc("Netty with protobuf");
        return builder.build();
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
        cause.printStackTrace();
        ctx.close();// An exception occurs and the link is closed}}Copy the code
The client

Let’s write the client startup class


public class SubReqClient {

    public void connect(int port, String host) throws Exception {
        // Configure the client NIO thread group
        EventLoopGroup group = new NioEventLoopGroup();
        try {
            Bootstrap b = new Bootstrap();
            b.group(group).channel(NioSocketChannel.class)
                    .option(ChannelOption.TCP_NODELAY, true)
                    .handler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        public void initChannel(SocketChannel ch)
                                throws Exception {
                            ch.pipeline().addLast(
                                    new ProtobufVarint32FrameDecoder());
                            ch.pipeline().addLast(
                                    new ProtobufDecoder(
                                            SubscribeRespProto.SubscribeResp
                                                    .getDefaultInstance()));
                            ch.pipeline().addLast(
                                    new ProtobufVarint32LengthFieldPrepender());
                            ch.pipeline().addLast(new ProtobufEncoder());
                            ch.pipeline().addLast(newSubReqClientHandler()); }});// Initiate an asynchronous connection
            ChannelFuture f = b.connect(host, port).sync();
            // The contemporary client link is down
            f.channel().closeFuture().sync();
        } finally {
            // Exit gracefully, freeing the NIO thread groupgroup.shutdownGracefully(); }}/ * * *@param args
     * @throws Exception
     */
    public static void main(String[] args) throws Exception {
        new SubReqClient().connect(8080."127.0.0.1"); }}Copy the code

Then we write the client side of the processing class

public class SubReqClientHandler extends ChannelInboundHandlerAdapter {

    public SubReqClientHandler(a) {}@Override
    public void channelActive(ChannelHandlerContext ctx) {
        for (int i = 0; i < 10; i++) {
            ctx.write(subReq(i));
        }
        ctx.flush();
    }

    private SubscribeReqProto.SubscribeReq subReq(int i) {
        SubscribeReqProto.SubscribeReq.Builder builder = SubscribeReqProto.SubscribeReq
                .newBuilder();
        builder.setSubReqID(i);
        builder.setUserName("netty");
        builder.setProductName("netty with protobuf");
        builder.setAddress("netty address");
        return builder.build();
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg)
            throws Exception {
        System.out.println("Receive server response : [" + msg + "]");
    }

    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
        ctx.flush();
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) { cause.printStackTrace(); ctx.close(); }}Copy the code
The end

This article briefly introduces the following contents:

  1. Google ProtobufWhat is the
  2. How to useGoogle ProtobufThe generated code
  3. NettyintegrationGoogle ProtobufThe code of

It’s not that Google Protobuf needs Netty to work with it, as you can see Protobuf can work with any framework or application on its own, and Netty is no different when it comes to integration as a generic POJO.

The end!