“This is the 8th day of my participation in the First Challenge 2022. For details: First Challenge 2022”

JSON has gradually replaced XML because it is lightweight, easy to read, and developer-friendly, so most of the front and back end interactions are now using JSON, but thanks to the rise of containerization and K8s. Protobuf, designed by Google, is slowly beginning to emerge as a cross-platform protocol for serializing structured data. At the same time, the application of gRPC for Protobuf has made the rapid rise of Protobuf. Many open source projects in China, such as Nacos, Dubbo3 and RocketMQ5, are based on gRPC. So why do we use JSON more often in Protobuf development since there are so many benefits? Analysis from the following aspects:

  • Time to serialize and deserialize
  • Deserialize memory footprint for the same structured data
  • Developer friendliness

1. Serialization and deserialization time

Define a simple User data structure, starting with the definition of Protobuf

syntax = "proto3";

option java_multiple_files = true;
option java_package = "com.github.mxsm.grpc.login";

message UserProto{

  string name = 1;

  int32 age = 2;

  string address = 3;

  string email = 4;

  string phone = 5;
}
Copy the code

Java class definition

public class UserJson {

    private String name;

    private int age;

    private String address;

    private String email;

    private String phone;
	// Omit the get set method

}
Copy the code

Write serialized tests

@BenchmarkMode(Mode.AverageTime)
@Warmup(iterations = 2, time = 5)
@Measurement(iterations = 3, time = 5)
@Threads(1)
@Fork(1)
@State(value = Scope.Benchmark)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public class ProtobufJsonTest {


    @Param({"100000","1000000"})
    private int counter;

    private UserJson userJson;

    private UserProto userProto;

    private byte[] userJsonBytes;

    private byte[] userProtoBytes;

    @Setup
    public void init(a){

        Faker faker = new Faker(new Locale("zh-CN"));
        String name = faker.name().fullName();
        int age = faker.number().numberBetween(1.100);
        String address = faker.address().fullAddress();
        String email = faker.internet().emailAddress();
        String phone = faker.phoneNumber().cellPhone();

        userJson = new UserJson();
        userJson.setName(name);
        userJson.setAge(age);
        userJson.setAddress(address);
        userJson.setEmail(email);
        userJson.setPhone(phone);

        userJsonBytes = JSON.toJSONBytes(userJson);

        userProto = UserProto.newBuilder().setName(name).setAge(age).setAddress(address).setEmail(email)
            .setPhone(phone).build();

        userProtoBytes = userProto.toByteArray();
    }


    @Benchmark
    public void protobufSerializable(Blackhole blackhole) {

        byte[] bytes = null;
        int i  = 0;
        for(; i < counter; ++i){ bytes = userProto.toByteArray(); } blackhole.consume(bytes); }@Benchmark
    public void jsonSerializable(Blackhole blackhole) {

        int i  = 0;
        byte[] bytes = null;
        for(; i < counter; ++i){ bytes = JSON.toJSONBytes(userJson); } blackhole.consume(bytes); }@Benchmark
    public void protobufDeserialization(Blackhole blackhole) throws InvalidProtocolBufferException {

        UserProto up = null;
        int i  = 0;
        for(; i < counter; ++i){ up = UserProto.parseFrom(userProtoBytes); } blackhole.consume(up); }@Benchmark
    public void jsonDeserialization(Blackhole blackhole) {

        int i  = 0;
        UserJson aaa = null;
        for(; i < counter; ++i){ aaa = JSON.parseObject(userJsonBytes, UserJson.class); } blackhole.consume(aaa); }public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder().
            include(ProtobufJsonTest.class.getSimpleName())
            .result("result.json")
            .resultFormat(ResultFormatType.JSON).build();
        newRunner(opt).run(); }}Copy the code

Tips: This test again used JMH and Faker, and Fastjson for JSON serialization and deserialization

Code running results:

As you can see from the figure above, protobuf serializes and deserializes better than JSON. (Different JSON serialization tools may have different speeds, but Fastjson is the faster serialization tool.)

2. Occupied memory

Compare the size of the serialized byte array. Test code:

public class MemeryTest {

    public static void main(String[] args) {
        Faker faker = new Faker(new Locale("zh-CN"));
        String name = faker.name().fullName();
        int age = faker.number().numberBetween(1.100);
        String address = faker.address().fullAddress();
        String email = faker.internet().emailAddress();
        String phone = faker.phoneNumber().cellPhone();
        System.out.println(name = ""+ age + "" + address + ""+ email+ ""+phone);
        UserJson  userJson = new UserJson();
        userJson.setName(name);
        userJson.setAge(age);
        userJson.setAddress(address);
        userJson.setEmail(email);
        userJson.setPhone(phone);

        byte[] userJsonBytes = JSON.toJSONBytes(userJson);

        UserProto userProto = UserProto.newBuilder().setName(name).setAge(age).setAddress(address).setEmail(email)
            .setPhone(phone).build();

        byte[] userProtoBytes = userProto.toByteArray();

        System.out.println("userJsonBytes="+userJsonBytes.length + "; userProtoBytes="+userProtoBytes.length); }}Copy the code

Running results:

Protobuf is 45 bytes less than JSON. A Protobuf takes up fewer bytes than a JSON sequence, giving it an advantage in network transmission.

3. Developer friendliness

Use MemeryTest to add two sentences at the end of the test:

System.out.println(new String(userJsonBytes));
System.out.println(new String(userProtoBytes));
Copy the code

Running results:

Found that JSON developers can recognize it well, while Protobuf is garbled. This illustrates a problem: JSON is better than Protobuf in terms of programmer friendliness.

In addition, there is a JSON and the corresponding Bean can be converted to each other, while Protobuf needs to define structured data in proto file first, and then convert it into the corresponding language data structure through compilation tools, which is cumbersome to use.

4. To summarize

  • Protobuf is faster at serialization and de-sequencing than JSON, and uses less memory after serialization than JSON. So protobuf in general is due to JSON
  • Because it is not as developer-friendly as JSON, the process is cumbersome. This may be why JSON is more popular than Protobuf, but Protobuf may be better suited for projects where performance and data transfer requirements are extremely high, such as IM data interaction.
  • Combining Protobuf with JSON might be a good option (this was not tested)