preface

Kryo is a highly effective Java serialization/deserialization library that is currently used by Twitter, Yahoo, Apache, Strom, and other big data sites such as Apache spark and Hive.

Why kryo and not others?

Because the performance is good enough. The only serialization library that is more efficient than Kyro is Google’s Protobuf (and the performance is very close). Protobuf has the disadvantage of generating a corresponding Proto file for each class structure to be transferred (it can also be stored in the same proto file, if extensibility is considered, Not recommended in a proto file), if a class changes, you must regenerate the proto file corresponding to that class; In addition, considering that the project uses all The Java technology stack, there is no compatibility problem between different programming languages, so kryo was finally adopted as the serialization library.

Usage scenarios

(Data exchange or data persistence) such as using Kryo to serialize objects into byte arrays and send them to message queues or put them into noSQL such as Redis.

Note: Since Kryo is not thread-safe, a simple wrapper design is required for use in multithreaded situations so that serialization and deserialization can be used in multithreaded situations safely

Serialization and deserialization interface design

/** * serialization tool (the program calls this interface to implement serialization/deserialization between obj<->byte[]) *@author eguid
 *
 */
public interface Serializer{
	
	/** * serialize *@param t
	 * @param bytes
	 */
	public void serialize(Object t,byte[] bytes);
	
	/** * serialize *@param obj
	 * @param bytes
	 * @param offset
	 * @param count
	 */
	public void serialize(Object obj, byte[] bytes, int offset, int count);
	
	/** * deserialize *@paramBytes-byte array *@return T<T>
	 */
	public <T>T deserialize(byte[] bytes);
	
 
	/** * deserialize *@param bytes
	 * @param offset
	 * @param count
	 * @return* /
	public <T>T deserialize(byte[] bytes, int offset, int count);
 
}
Copy the code

Use Kryo to implement the above interface

/** * Kyro-based serialization/deserialization tool **@author eguid
 *
 */
public class kryoSerializer implements Serializer {
 
	// Since Kryo is not thread-safe, each thread uses a separate Kryo
	final ThreadLocal<Kryo> kryoLocal = new ThreadLocal<Kryo>() {
		@Override
		protected Kryo initialValue(a) {
			Kryo kryo = new Kryo();
			kryo.register(ct, new BeanSerializer<>(kryo, ct));
			returnkryo; }};final ThreadLocal<Output> outputLocal = new ThreadLocal<Output>();
	final ThreadLocal<Input> inputLocal = new ThreadLocal<Input>();
	privateClass<? > ct =null;
 
	public kryoSerializer(Class
        ct) {
		this.ct = ct;
	}
 
	publicClass<? > getCt() {return ct;
	}
 
	public void setCt(Class
        ct) {
		this.ct = ct;
	}
 
	@Override
	public void serialize(Object obj, byte[] bytes) {
		Kryo kryo = getKryo();
		Output output = getOutput(bytes);
		kryo.writeObjectOrNull(output, obj, obj.getClass());
		output.flush();
	}
 
	@Override
	public void serialize(Object obj, byte[] bytes, int offset, int count) {
		Kryo kryo = getKryo();
		Output output = getOutput(bytes, offset, count);
		kryo.writeObjectOrNull(output, obj, obj.getClass());
		output.flush();
	}
 
	/** * get kryo **@param t
	 * @return* /
	private Kryo getKryo(a) {
		return kryoLocal.get();
	}
 
	/** * gets Output and sets the initial array **@param bytes
	 * @return* /
	private Output getOutput(byte[] bytes) {
		Output output = null;
		if ((output = outputLocal.get()) == null) {
			output = new Output();
			outputLocal.set(output);
		}
		if(bytes ! =null) {
			output.setBuffer(bytes);
		}
		return output;
	}
 
	/** * get Output **@param bytes
	 * @return* /
	private Output getOutput(byte[] bytes, int offset, int count) {
		Output output = null;
		if ((output = outputLocal.get()) == null) {
			output = new Output();
			outputLocal.set(output);
		}
		if(bytes ! =null) {
			output.writeBytes(bytes, offset, count);
		}
		return output;
	}
 
	/** * get Input **@param bytes
	 * @param offset
	 * @param count
	 * @return* /
	private Input getInput(byte[] bytes, int offset, int count) {
		Input input = null;
		if ((input = inputLocal.get()) == null) {
			input = new Input();
			inputLocal.set(input);
		}
		if(bytes ! =null) {
			input.setBuffer(bytes, offset, count);
		}
		return input;
	}
 
	@SuppressWarnings("unchecked")
	@Override
	public <T> T deserialize(byte[] bytes, int offset, int count) {
		Kryo kryo = getKryo();
		Input input = getInput(bytes, offset, count);
		return (T) kryo.readObjectOrNull(input, ct);
	}
 
	@Override
	public <T> T deserialize(byte[] bytes) {
		return deserialize(bytes, 0, bytes.length);
	}
Copy the code

Test kryo’s serialization and deserialization

Why nanoseconds instead of milliseconds? Unlike Java’s native serialization and deserialization, which can take milliseconds, Kryo serialization and deserialization are fast, with individual objects being serialized and deserialized in 0.0x milliseconds (or faster with a better computer)

Serializer ser = new kryoSerializer(Msg.class);
		for (int i = 0; i < 10; i++) {
 
			Msg msg = new Msg();
 
			msg.setVersion_flag(new byte[] { 1.2.3 });
			msg.setCrc_code((short) 1);
			msg.setMsg_body(new byte[] { 123.123.123.43.42.1.12.45.57.98 });
			byte[] bytes = new byte[300];
			long start = System.nanoTime();
			ser.serialize(msg, bytes);
			System.err.println("Serialization time:" + (System.nanoTime() - start));
			System.out.println(msg);
			System.out.println(Arrays.toString(bytes));
 
			Msg newmsg = null;
			start = System.nanoTime();
			newmsg = ser.deserialize(bytes);
			System.err.println("Deserialization time:" + (System.nanoTime() - start));
			System.out.println(newmsg);
		}
Copy the code

—-end—-