How to use Protobuf in the front end

How to use protobuf (Vue) in the front end, node has been delayed by lazy cancer. Last time, there was a lot of discussion about why protobuf is used on the front end. Our company uses Protobuf for ios, Android and Web. I have also mentioned some benefits and benefits of protobuf in the previous sharing. If your company uses it, or may use it in the future, these two posts may give you some inspiration.

Analytical thinking

The protobuf.js library is also used for parsing.

As mentioned earlier, in vue, to avoid using.proto files directly, you need to package all.proto files into.js.

On the Node side, it can also be packaged into JS files for processing. However, node is a server environment, which allows. Proto to exist, so we can actually use it in an elegant way: directly parsing.

The desired effect

Encapsulate two basic modules:

• request.js: Initiates a request based on the interface name, request body, and return value type.
• proto.jsUsed to parse proTO and convert data to binary. It can be used in projects like this:

// lib/api.js const request = require('./request')
const proto = require('./proto') /** ** @param {* request data} params * getStudentList is the interface name * school.PBStudentListRsp is defined return model * school.PBStudentListReq Is the defined request body model */ exports.getstudentList =function getStudentList (params) {
  const req = proto.create('school.PBStudentListReq', params)
  return request('school.getStudentList', req, 'school.PBStudentListRsp'} // use lib/api.js const API = require('.. /lib/api')
const req = {
  limit: 20,
  offset: 0
}
api.getStudentList(req).then((res) => {
  console.log(res)
}).catch(() => {
  // ...
})
Copy the code

Preparations:

Prepare a copy of how to use protobuf (vue) in the front end. Proto, note that this proto defines two namespaces: framework and School. Proto file source code

Encapsulation proto. Js

See the official documentation for how to convert object to buffer:

protobuf.load("awesome.proto".function(err, root) {
    if (err)
        throw err;
    var AwesomeMessage = root.lookupType("awesomepackage.AwesomeMessage");

    var payload = { awesomeField: "AwesomeString" };

    var message = AwesomeMessage.create(payload); 

    var buffer = AwesomeMessage.encode(message).finish();
});
Copy the code

It should be easy to understand: load awesome. Proto, and then convert the data payload into the buffer we want. Create and encode are methods provided by ProtobufJS.

If we only have a.proto file in our project, we can use it as an official document. But in real projects, there are often many.proto files, if each PBMessage needs to know which.proto file is in, it will be difficult to use, so you need to wrap it. The enumeration of interfaces given to us by the server side is generally like this:

getStudentList = 0; // Get the list of all students, PBStudentListReq => PBStudentListRspCopy the code

We just tell you that the request body of this interface is PBStudentListReq, and the return value is PBStudentListRsp, and we don’t know which.proto file they’re in.

For ease of use, we want to encapsulate a method of the form:

const reqBuffer = proto.create('school.PBStudentListReq', dataObj) 
Copy the code

You only need to use PBStudentListReq and dataObj as parameters, do not care which PBStudentListReq is in the. Proto file. Here’s the tricky part: how do you find it by type? Proto?

The way to do this is to put all the.proTos in memory, and then get the corresponding type based on the name.

Write a loadProtoDir method that stores all protos in the _proTO variable.

// proto.js
const fs = require('fs')
const path = require('path')
const ProtoBuf = require('protobufjs')

let_proTO = null // Store all. Proto in _protofunction loadProtoDir (dirPath) {
  const files = fs.readdirSync(dirPath)

  const protoFiles = files
    .filter(fileName => fileName.endsWith('.proto'))
    .map(fileName => path.join(dirPath, fileName))
  _proto = ProtoBuf.loadSync(protoFiles).nested
  return _proto
}
Copy the code

_proto is like a tree, and we can walk through the tree to find the type, or we can get it directly from other methods, such as lodash.get(), which supports values like obj[‘xx.xx.xx’].

const _ = require('lodash')
const PBMessage = _.get(_proto, 'school.PBStudentListReq')
Copy the code

In this way, we can successfully obtain PBMessage in all proto according to the type. PBMessage will have create, encode and other methods provided by Protobuf. js, which we can directly use and convert object into buffer.

const reqData = {a: '1'}
const message = PBMessage.create(reqData)
const reqBuffer = PBMessage.encode(message).finish()
Copy the code

To make it easier to use, encapsulate it into three functions:

let_proTO = null // Store all. Proto in _protofunction loadProtoDir (dirPath) {
  const files = fs.readdirSync(dirPath)

  const protoFiles = files
    .filter(fileName => fileName.endsWith('.proto'))
    .map(fileName => path.join(dirPath, fileName))
  _proto = ProtoBuf.loadSync(protoFiles).nested
  return_proto} // BasedtypeThe Name for PBMessagefunction lookup (typeName) {
  if(! _.isString(typeName)) {
    throw new TypeError('typeName must be a string')}if(! _proto) { throw new TypeError('Please load proto before lookup')}return _.get(_proto, typeName)
}

functionCreate (protoName, obj) {const model = lookup(protoName) const model = lookup(protoName)if(! model) { throw new TypeError(`${protoName} not found, please check it again`)
  } 
  const req = model.create(obj)
  returnModel.encode (req).finish()} module.exports = {lookup, loadProtoDir}Copy the code

This requires loadProtoDir to place all proTos into memory before using create and lookup.

Packaging request. Js

It is recommended to first look at messageType. proto, which defines the interface enumeration, request body, and response body with the back-end convention.

const rp = require('request-promise') 
const proto = require('./proto.js'Proto.js /** ** @param {* interface name} msgType * @param {* buffer after proto.create()} requestBody * @param {* return type} responseType */functionRequest (msgType, requestBody, responseType) {const _msgType = proto.lookup('framework.PBMessageType'[msgType] // PBMessageRequest is a common request body that carries some additional token information, etc., and is passed by the back endtypeConst PBMessageRequest = proto.lookup('framework.PBMessageRequest')
  const req = PBMessageRequest.encode({
    timeStamp: new Date().getTime(),
    type: _msgType,
    version: '1.0',
    messageData: requestBody,
    token: 'xxxxxxx'}).finish() // Initiate the request. In vue we can use axios to initiate ajax, but we need to change the node side, for example"request"// I recommend using a good library here:"request-promise", which supports promise const options = {method:'POST',
    uri: 'http://your_server.com/api',
    body: req,
    encoding: null,
    headers: {
      'Content-Type': 'application/octet-stream'}}returnRp.post (options).then((res) => {const decodeResponse = proto.lookup()'framework.PBMessageResponse').decode(res)
    const { resultInfo, resultCode } = decodeResponse
    if(resultCode === 0) {PBMessageResponse messageData const Model = proto.lookup(responseType)let msgData = model.decode(decodeResponse.messageData)
      return msgData
    } else {
      throw new Error(`Fetch ${msgType} failed.`)
    }
  })
}

module.exports = request
Copy the code

use

Request. js and proto.js provide the underlying services. For ease of use, we also need to encapsulate an api.js that defines all the apis in the project.

const request = require('./request')
const proto = require('./proto')

exports.getStudentList = function getStudentList (params) {
  const req = proto.create('school.PBStudentListReq', params)
  return request('school.getStudentList', req, 'school.PBStudentListRsp')}Copy the code

When using an interface in a project, just require(‘lib/ API ‘) and don’t reference proto.js and request.js directly.

// test.js

const api = require('.. /lib/api')

const req = {
  limit: 20,
  offset: 0
}
api.getStudentList(req).then((res) => {
  console.log(res)
}).catch(() => {
  // ...
})
Copy the code

The last

The demo source code