Moment For Technology

A high-performance, small and beautiful serialization tool!

Posted on Dec. 2, 2022, 7:39 p.m. by Susan Monk
Category: The back-end Tag: The back-end java

By Fredalxin address: Fredal. xin/ Kryo-quicks...

Kryo is a high performance serialization/deserialization tool. Due to its variable length storage and bytecode generation mechanism, high speed and small size, Kryo is an alternative to Json and Protobuf in some scenarios.

Rely on

First, we introduce maven dependencies:

dependency
    groupIdcom.esotericsoftware/groupId
    artifactIdkryo/artifactId
    version4.02./version
/dependency
Copy the code

Note that as Kryo uses a higher version of ASM, it is a common problem that it may conflict with the ASM that the business already relies on. Just change the dependency to:

dependency
    groupIdcom.esotericsoftware/groupId
    artifactIdkryo-shaded/artifactId
    version4.02./version
/dependency
Copy the code

Record type Information

This is a feature of Kryo that you can write object information directly to the serialized data. When you deserialize, you can find the original Class information exactly without error. This means you don't need to pass Class or Type information when you write readXXX.

Accordingly, Kryo provides two ways to read and write. Record type information writeClassAndObject/readClassAndObject method, as well as the traditional writeObject/readObject methods.

Thread safety

Kryo's objects themselves are not thread-safe, so we have two options to ensure thread-safe.

Use Threadlocal to ensure thread safety:

private static final ThreadLocalKryo kryoLocal = new ThreadLocalKryo() {
	protected Kryo initialValue(a) {
		Kryo kryo = new Kryo();
        kryo.setInstantiatorStrategy(new Kryo.DefaultInstantiatorStrategy(
                    new StdInstantiatorStrategy()));
		return kryo;
	};
};
Copy the code

Or use the pool provided by Kryo:

public KryoPool newKryoPool(a) {
    return new KryoPool.Builder(() - {
        final Kryo kryo = new Kryo();
        kryo.setInstantiatorStrategy(new Kryo.DefaultInstantiatorStrategy(
            new StdInstantiatorStrategy()));
        return kryo;
    }).softReferences().build();
}
Copy the code

Instantiate the unit

In the above notice kryo. SetInstantiatorStrategy (new kryo. DefaultInstantiatorStrategy (new StdInstantiatorStrategy ())); This statement shows that the instantiator is specified.

In some open source software that relies on Kryo, null-pointer exceptions may be thrown due to a problem with the instantiator specification. For example, in some versions of Hive, StdInstantiatorStrategy is specified by default.

public static ThreadLocalKryo runtimeSerializationKryo = new ThreadLocalKryo() {
    @Override
    protected synchronized Kryo initialValue(a) {
        Kryo kryo = new Kryo();
        kryo.setClassLoader(Thread.currentThread().getContextClassLoader());
        kryo.register(java.sql.Date.class, new SqlDateSerializer());
        kryo.register(java.sql.Timestamp.class, new TimestampSerializer());
        kryo.register(Path.class, new PathSerializer());
        kryo.setInstantiatorStrategy(newStdInstantiatorStrategy()); .return kryo;
    };
};
Copy the code

While StdInstantiatorStrategy creates objects according to JVM version information and JVM vendor information, it can create objects without calling any constructor of the object.

When you run into objects like An ArrayList, for example, you have a problem. Take a look at the source code for ArrayList:

public ArrayList(a) {
    this.elementData = DEFAULTCAPACITY_EMPTY_ELEMENTDATA;
}
Copy the code

Since the constructor is not called, here elementData will be NULL and an exception will be thrown when a method like ensureCapacity is called.

 public void ensureCapacity(int minCapacity) {
     if (minCapacity  elementData.length
          !(elementData == DEFAULTCAPACITY_EMPTY_ELEMENTDATA
               minCapacity = DEFAULT_CAPACITY)) {
         modCount++;
         grow(minCapacity);
     }
 }
Copy the code

Solution is very simple, such as the framework of code written as specified instantiation, the first to use the default no DefaultInstantiatorStrategy structure strategy, if use StdInstantiatorStrategy again failed to create object.

Class registration

When Kryo writes an instance of an object, the default is to write the fully qualified name of the class. It is inefficient to write the class name along with the serialized data, so Kryo supports class registration for optimization.

kryo.register(SomeClassA.class);
kryo.register(SomeClassB.class);
kryo.register(SomeClassC.class);
Copy the code

Registration gives each class an INT Id associated with it. This is obviously more efficient than the class name, but it also requires that the Id of deserialization be the same as that of serialization. This means that the order of registration is very important.

However, due to practical reasons, the same code, the same Class registration number on different machines can not be guaranteed to be consistent, so the deserialization may be a problem when deployed on multiple machines.

So kryo default would ban class registration, of course, if you want to open this property, can pass kryo. SetRegistrationRequired (true); To open.

A circular reference

This is support for circular references, which prevent stack overruns, and is turned on by kryo by default. Kryo. setReferences(false) is used when you are sure that no cyclic reference will occur; Turn off circular reference detection to improve some performance.

Variable length storage

Kryo adopts the mechanism of variable length storage for both int and long types. Taking int as an example, it generally needs 4 bytes to store, while Kryo can store by 1-5 variable length bytes, so as to avoid the waste of 0 high bytes.

A maximum of 5 bytes of storage is required because, in the variable-length int procedure, only 7 of the 8 bits of a byte are used to store significant digits, and the highest bit is used to mark whether the next byte needs to be read. 1 means yes, and 0 means no.

There are also applications of variable-length storage in string storage. The overall structure of string serialization is length+ content, so Length will also write the length of the character using variable-length int.

Scenarios used with caching

In real world development, class adding and deleting fields is common, but for Kryo, it is not supported, and if you happen to need caching, the problem will be magnified.

For example, if an object is serialized using Kryo and the data is in the cache, the cache will report an error when the object is deserialized if a property is added or deleted. So frequent use of caching scenarios, you can avoid kryo as much as possible.

But now the Kryo to provide the support of compatibility, using CompatibleFieldSerializer. Class, in the Kryo. WriteClassAndObject time written information is as follows:

class name|field length|field1 name|field2 name|field1 value| filed2 value
Copy the code

When kryo.readClassandobject is read, field Names are read first, and the result is constructed by matching the field and order of the currently deserialized class.

Of course, none of this matters if you do a good job of cache isolation.

Recent hot articles recommended:

1.1,000+ Java interview questions and answers (the latest version of 2021)

2. Finally got the IntelliJ IDEA activation code through the open source project, how sweet!

3. Ali Mock tool officially open source, kill all the Mock tools on the market!

4.Spring Cloud 2020.0.0 is officially released, a new and disruptive version!

5. "Java Development Manual (Songshan version)" the latest release, speed download!

Feel good, don't forget to click "like" + forward oh!

Search
About
mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.