introduce

ProtoBuf is a tool developed by the Google team for efficiently storing and reading structured data. What is structured data, literally, is data with a certain structure. For example, there are many records in the phone book, and each entry contains name, ID, email, phone, etc. This structure is repeated.

The similar

XML and JSON can also be used to store this type of structured data, but the data represented using ProtoBuf is much more efficient and compactly smaller.

The principle of

ProtoBuf is a unique.proto that is independent of the programming language through the ProtoBuf compiler The suffixed data structure files are compiled into class files specific to each programming language (Java,C/C++,Python), and then the API can be called through the supporting library lib provided by Google for each programming language. (See the documentation on how to write the proto structure.)

ProtoBuf compiler installed

Mac : brew install protobuf

For example

1. Create a proto file called message.proto

syntax = "proto3"; message Person { int32 id = 1; string name = 2; repeated Phone phone = 4; enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; } message Phone { string number = 1; PhoneType type = 2; }}

2. Create a Java project and place the proto file in SRC /main/proto folder

3. Compile the proto file into the SRC /main directory using the command line CD

Protoc –java_out=./ Java./proto/*.proto

You’ll notice that the corresponding Java class has been generated in your SRC /main/ Java file

Here’s just one example of how Gradle uses dependencies

Implementation 'com. Google. Protobuf: protobuf - Java: 3.9.1'

5. Convert Java objects to ProtoBuf data

Message.Person.Phone.Builder phoneBuilder = Message.Person.Phone.newBuilder(); Message.Person.Phone phone1 = phoneBuilder .setNumber("100860") .setType(Message.Person.PhoneType.HOME) .build(); Message.Person.Phone phone2 = phoneBuilder .setNumber("100100") .setType(Message.Person.PhoneType.MOBILE) .build(); Message.Person.Builder personBuilder = Message.Person.newBuilder(); personBuilder.setId(1994); personBuilder.setName("XIAOLEI"); personBuilder.addPhone(phone1); personBuilder.addPhone(phone2); Message.Person person = personBuilder.build(); long old = System.currentTimeMillis(); byte[] buff = person.toByteArray(); System.out.println("ProtoBuf time: "+ (System.currentTimeMillis() -old)); System.out.println(Arrays.toString(buff)); System.out.println("ProtoBuf data length :" + buff.length);

6. Convert the ProtoBuf data back to Java objects

System.out.println(" -start decoding -"); old = System.currentTimeMillis(); Message.Person personOut = Message.Person.parseFrom(buff); System.out.println("ProtoBuf decoding time: "+ (System.currentTimeMillis() -old)); System.out.printf("Id:%d, Name:%sn", personOut.getId(), personOut.getName()); List<Message.Person.Phone> phoneList = personOut.getPhoneList(); For (message.person. Phone Phone: phoneList) {system.out.printf (" Phone number :%s (%s)n", phone.getNumber(), phone.getType()); }

To compare

To demonstrate ProtoBuf’s advantages, I wrote a Java class with the same structure and converted Java objects into JSON data to compare with ProtoBuf. The JSON compiler library uses the GSON library provided by Google, so parts of the JSON code are not posted and the results are displayed directly

Comparison Results Results

Run 1 time

JSON data length :106 - start decoding - JSON decoding once, time: 1ms Data Length :34 - Start decoding - Protobuf decoding once, Time: 3ms

Run ten times

JSON data length :106 - start decoding - JSON decoding 10 times, time: 4ms 29ms ProtoBuf data length :34 - start decoding - ProtoBuf decoding 10 times, time: 3ms

Run 100 times

JSON data length :106 - start decoding - JSON decode 100 times, time: 8ms 31ms ProtoBuf data length :34 - start decoding - ProtoBuf decoding 100 times, time: 4ms

Run 1000 times

JSON data length :106 - start decoding - JSON decoding 1000 times, time: 21ms 37ms ProtoBuf data length :34 - start decoding - ProtoBuf decoding 1000 times, time: 8ms

Run it 10,000 times

JSON data length :106 - start decoding - JSON decode 10000 times, time: 93ms 49ms ProtoBuf data length :34 - start decoding - ProtoBuf decoding 10000 times, time: 23ms

Run it 100,000 times

JSON data length :106 - start decoding - JSON decoding 100000 times, time: 248ms Data Length :34 - Start decoding - Protobuf decoded 100,000 times, Time: 58ms

conclusion

Codec performance of the above chestnut is just a simple sampling, in fact according to my experiment found

  • Under 1,000 times, ProtoBuf’s encoding and decoding performance is comparable to, or even worse than, JSON.
  • The number of times in more than 2,000, ProtoBuf encoding decoding performance, are much higher than JSON.
  • When the number of times is over 100,000, the encoding and decoding performance of ProtoBuf is obvious, far higher than that of JSON.

Protobuf has a memory footprint of 34, whereas JSON has a memory footprint of 106, so ProtoBuf has a memory footprint of 1/3 JSON’s.

At the end

There’s a lot to optimize in this experiment, but even in this rough test, you can see the advantages of ProtoBuf.

Compatible with

The new field

  • Add a NICKNAME field to the proto file
  • Generating Java Files
  • Using the old proto byte array data, convert it to an object
ID :1994, NAME: Xiaolei Mobile Number :100860 (HOME) Mobile Number :100100 (MOBILE) getNICKNAME =

As a result, the conversion can be successful.

Delete the field

  • Delete the name field in the proto file
  • Generating Java Files
  • Using the old proto byte array data, convert it to an object
ID :1994, NAME: NULL Mobile number :100860 (HOME)

As a result, the conversion can be successful.

Source: my.oschina.net/xiaolei123/blog/3085607



Welcome to pay attention to my WeChat official account “Code Farmer Breakthrough”, share Python, Java, big data, machine learning, artificial intelligence and other technologies, pay attention to the improvement of code farmer technology, career breakthrough, thinking leap, 200,000 + code farmer growth charging first station, grow up with you who have dreams