The background,
I accidentally changed the package path of a class while maintaining an underlying common component. Unfortunately, this class is referenced and passed behind the facade by the businesses. Fortunately, the same class, the provider and consumer package path is inconsistent, does not cause each business error.
With curiosity, I have learned the debugging of Dubbo for several times. I would like to share some learning experience here.
1.1 Love and Hate of RPC
One of the advantages of Dubbo, the RPC framework for the Java language, is that it shields the call details and allows you to invoke remote services as if they were local methods, without having to worry about data formats. It is this feature that introduces some problems.
For example, jar package conflicts occur after the facade package is introduced, services cannot be started, a class cannot be found after the facade package is updated, and so on. The introduction of JAR packages results in some degree of coupling between consumers and providers.
It is this coupling that the provider is habitually expected to raise an error when it does not modify the path to the Facade package class. At first it seemed strange, but on reflection it was only right that the caller could complete the communication with the provider based on the agreed format and protocol. The provider’s own context information should not be concerned. The next step is to uncover the Dubbo encoding and decoding process.
Dubbo codec
Dubbo uses NetTY as the default communication framework, and all analysis is based on NetTY. The source code involved is dubbo-2.7.x version. In practice, a service is likely to be both a consumer and a provider. To simplify the combing process, assume that all are pure consumers and providers.
2.1 In Dubbo
Borrowing a picture from the official document of Dubbo, the communication and serialization layers are defined in the document, and the meaning of “codec” is not defined. Here, a brief explanation of “codec” is given.
Codec = DuBBo internal codec link + serialization layer
This article aims to tease out the conversions between the two data formats, from Java objects to binary streams and from binary streams to Java objects. For this purpose, in order to facilitate understanding, additional communication layer content, with encode and decode as the entrance, combing dubbo processing link. Dubbo is defined as Encoder, Decoder, so it is defined as “Decoder”.
2.2 the entrance
When the consumer initiates a connection with the NettyClient#doOpen method and initialses BootStrap, a different type of ChannelHandler is added to the Netty pipeline, including the codec.
Similarly, the provider provides the service in the NettyServer#doOpen method, which adds the codec when it initializes ServerBootstrap. (Adapter.getdecoder () -decoder, adapater.getencoder () -coder). NettyClient
/**
* Init bootstrap
*
* @throws Throwable
*/
@Override
protected void doOpen(a) throws Throwable {
bootstrap = new Bootstrap();
// ...
bootstrap.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
// ...
ch.pipeline()
.addLast("decoder", adapter.getDecoder())
.addLast("encoder", adapter.getEncoder())
.addLast("client-idle-handler".new IdleStateHandler(heartbeatInterval, 0.0, MILLISECONDS))
.addLast("handler", nettyClientHandler);
// ...}}); }Copy the codeNettyServer
/**
* Init and start netty server
*
* @throws Throwable
*/
@Override
protected void doOpen(a) throws Throwable {
bootstrap = new ServerBootstrap();
// ...
bootstrap.group(bossGroup, workerGroup)
.channel(NettyEventLoopFactory.serverSocketChannelClass())
.option(ChannelOption.SO_REUSEADDR, Boolean.TRUE)
.childOption(ChannelOption.TCP_NODELAY, Boolean.TRUE)
.childOption(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT)
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
// ...
ch.pipeline()
.addLast("decoder", adapter.getDecoder())
.addLast("encoder", adapter.getEncoder())
.addLast("server-idle-handler".new IdleStateHandler(0.0, idleTimeout, MILLISECONDS))
.addLast("handler", nettyServerHandler); }});// ...
}
Copy the code2.3 Consumer Link
Consumers encode when they send a message and decode when they receive a response.
Send a message
ChannelInboundHandler ... NettyCodecAdapter#getEncoder() ->NettyCodecAdapter$InternalEncoder#encode ->DubboCountCodec#encode ->DubboCodec#encode ->ExchangeCodec#encode ->ExchangeCodec#encodeRequest DubboCountCodec class actually references DubboCodec, since DubboCodec inherits from ExchangeCodec, The encode method is not overridden, so the actual code jump goes directly to the Exchange Dec# encode methodCopy the codeReceive the response
NettyCodecAdapter#getDecoder() ->NettyCodecAdapter$InternalDecoder#decode ->DubboCountCodec#decode ->DubboCodec#decode ->ExchangeCodec#decode ->DubboCodec#decodeBody ... MultiMessageHandler#received ->HeartbeatHadnler#received ->AllChannelHandler#received ... ChannelEventRunnable#run ->DecodeHandler#received DecodeHandler#decode ->DecodeableRpcResult#decode decode link is relatively complex, In the process of doing two decodes, DubboCodec#decodeBody, did not actually decode the channel data, but build DecodeableRpcResult object, and then in the business process of the Handler through the asynchronous thread to actually decode.Copy the code2.4 Providing End Links
The provider decodes the message when it receives it and encodes the response when it replies. Receives the message
NettyCodecAdapter#getDecoder() ->NettyCodecAdapter$InternalDecoder#decode ->DubboCountCodec#decode ->DubboCodec#decode ->ExchangeCodec#decode ->DubboCodec#decodeBody ... MultiMessageHandler#received ->HeartbeatHadnler#received ->AllChannelHandler#received ... ChannelEventRunnable#run ->DecodeHandler#received ->DecodeHandler#decode ->DecodeableRpcInvocation#decode The DecodeableRpcResult DecodeableRpcInvocation is replaced with the DecodeableRpcInvocation on the provider side. Reflects the Dubbo code in the good design, abstract processing link, shielding processing details, clear reusable process.Copy the coderesponses
NettyCodecAdapter#getEncoder() ->NettyCodecAdapter$InternalEncoder#encode ->DubboCountCodec#encode ->DubboCodec#encode ->ExchangeCodec#encode ->ExchangeCodec#encodeResponse and consumer send message link consistent, the difference is that the last step to distinguish Request and Response, different content encodingCopy the code2.5 Dubbo protocol header
Dubbo supports a variety of communication protocols, such as Dubbo protocol, HTTP, RMI, WebService and so on. The default is Dubbo. As a communication protocol, there are certain protocol formats and conventions, but these information is not concerned by services. It’s the Dubbo framework that adds and parses during coding.
Dubbo uses a fixed-length header and an indeterminate length body for data transmission. Here is the format definition of the header
**2byte: **magic, similar to the magic number in Java bytecode files, used to identify whether it is a dubbo protocol packet.
**1byte: ** message flag bit, 5 bits serialized ID, 1 bit heartbeat or normal request, 1 bit bidirectional or unidirectional, 1 bit request or response;
1 byte: * * * * Response status, specific type see com. Alibaba. Dubbo. Remoting. Exchange. The Response;
**8byte: ** message ID, the unique identification ID of each request;
**4byte: indicates the length of the message body.
For example, when a consumer sends a message, the code for setting the content of the message header is described in ExchangeCodec#encodeRequest.
Message coding
protected void encodeRequest(Channel channel, ChannelBuffer buffer, Request req) throws IOException {
Serialization serialization = getSerialization(channel);
// header.
byte[] header = new byte[HEADER_LENGTH];
// set magic number.
Bytes.short2bytes(MAGIC, header);
// set request and serialization flag.
header[2] = (byte) (FLAG_REQUEST | serialization.getContentTypeId());
if (req.isTwoWay()) {
header[2] |= FLAG_TWOWAY;
}
if (req.isEvent()) {
header[2] |= FLAG_EVENT;
}
// set request id.
Bytes.long2bytes(req.getId(), header, 4);
// encode request data.
int savedWriteIndex = buffer.writerIndex();
buffer.writerIndex(savedWriteIndex + HEADER_LENGTH);
ChannelBufferOutputStream bos = new ChannelBufferOutputStream(buffer);
ObjectOutput out = serialization.serialize(channel.getUrl(), bos);
if (req.isEvent()) {
encodeEventData(channel, out, req.getData());
} else {
encodeRequestData(channel, out, req.getData(), req.getVersion());
}
out.flushBuffer();
if (out instanceof Cleanable) {
((Cleanable) out).cleanup();
}
bos.flush();
bos.close();
int len = bos.writtenBytes();
checkPayload(channel, len);
// body length
Bytes.int2bytes(len, header, 12);
// write
buffer.writerIndex(savedWriteIndex);
buffer.writeBytes(header); // write header.
buffer.writerIndex(savedWriteIndex + HEADER_LENGTH + len);
}
Copy the codeThird, Hessian2
This section takes a closer look at the details of object serialization.
Dubbo supports multiple serialization formats, hessian2, JSON, JDK serialization, etc. Hessian2 is a modified version of Hessian by Ali and is the default serialization framework for Dubbo. In this case, the consumer side sends the message serialization object, receives the response deserialization as a case, looks at the processing details of Hessian2, while answering the introduction question.
3.1 the serialization
As mentioned earlier, the request encoding method is in Exchange code #encodeRequest, where the object data is serialized as Dubbocode #encodeRequestData DubboCodec
@Override
protected void encodeRequestData(Channel channel, ObjectOutput out, Object data, String version) throws IOException {
RpcInvocation inv = (RpcInvocation) data;
out.writeUTF(version);
// https://github.com/apache/dubbo/issues/6138
String serviceName = inv.getAttachment(INTERFACE_KEY);
if (serviceName == null) {
serviceName = inv.getAttachment(PATH_KEY);
}
out.writeUTF(serviceName);
out.writeUTF(inv.getAttachment(VERSION_KEY));
out.writeUTF(inv.getMethodName());
out.writeUTF(inv.getParameterTypesDesc());
Object[] args = inv.getArguments();
if(args ! =null) {
for (int i = 0; i < args.length; i++) {
out.writeObject(encodeInvocationArgument(channel, inv, i));
}
}
out.writeAttachments(inv.getObjectAttachments());
}
Copy the codeWe know that the Dubbo Invocation is stored in context. The version number, service name, method name, method parameter, return value and so on are first written here. The argument list is then iterated through, serializing each argument. In this case, the OUT object is the concrete serialization framework object, which defaults to Hessian2ObjectOutput. The OUT object is passed in as a parameter.
So where do you validate the actual serialized object?
Looking at the encoded call link from scratch, Exchangecode #encodeRequest has the following code:
ExchangeCodec
protected void encodeRequest(Channel channel, ChannelBuffer buffer, Request req) throws IOException {
Serialization serialization = getSerialization(channel);
// ...
ObjectOutput out = serialization.serialize(channel.getUrl(), bos);
if (req.isEvent()) {
encodeEventData(channel, out, req.getData());
} else {
encodeRequestData(channel, out, req.getData(), req.getVersion());
}
// ...
}
Copy the codeThe Out object comes from the Serialization object, along the way. In the CodecSupport class there is the following code:
CodecSupport
public static Serialization getSerialization(URL url) {
return ExtensionLoader.getExtensionLoader(Serialization.class).getExtension(
url.getParameter(Constants.SERIALIZATION_KEY, Constants.DEFAULT_REMOTING_SERIALIZATION));
}
Copy the codeAs you can see, the Serialization object is selected based on the SPI of Dubbo using the URL information, which defaults to hessian2. Serialization.serialize (channel.geturl (),bos)
Hessian2Serialization
@Override
public ObjectOutput serialize(URL url, OutputStream out) throws IOException {
return new Hessian2ObjectOutput(out);
}
Copy the codeSo far, the actual serialized object has been found, and the parameter serialization logic is relatively simple and will not be described further, as follows: write request parameter type → write parameter field name → iteration field list, field serialization.
3.2 Deserialization
Deserialization has more constraints than serialization. When serializing an object, you don’t need to care about the recipient’s actual data format. Deserialization, however, requires that the raw data match the object. The raw data could be a binary stream or JSON.
As mentioned in the consumer decoding link, two decodes occurred, the first time did not actually decode business data, but converted to DecodeableRpcResult. The specific code is as follows:
DubboCodec
@Override
protected Object decodeBody(Channel channel, InputStream is, byte[] header) throws IOException {
byte flag = header[2], proto = (byte) (flag & SERIALIZATION_MASK);
// get request id.
long id = Bytes.bytes2long(header, 4);
if ((flag & FLAG_REQUEST) == 0) {
// decode response...
try {
DecodeableRpcResult result;
if (channel.getUrl().getParameter(DECODE_IN_IO_THREAD_KEY, DEFAULT_DECODE_IN_IO_THREAD)) {
result = new DecodeableRpcResult(channel, res, is,
(Invocation) getRequestData(id), proto);
result.decode();
} else {
result = new DecodeableRpcResult(channel, res,
new UnsafeByteArrayInputStream(readMessageData(is)),
(Invocation) getRequestData(id), proto);
}
data = result;
} catch (Throwable t) {
// ...
}
return res;
} else {
// decode request...
returnreq; }}Copy the codeThe key point
1) The decoder request or decoder response is distinguished. For the consumer, it is the decoder response. For the provider, this is the decoding request.
2) Why does it happen twice? See this line for details:
if (channel.getUrl().getParameter(DECODE_IN_IO_THREAD_KEY, DEFAULT_DECODE_IN_IO_THREAD)) {
inv = new DecodeableRpcInvocation(channel, req, is, proto);
inv.decode();
} else {
inv = new DecodeableRpcInvocation(channel, req,
new UnsafeByteArrayInputStream(readMessageData(is)), proto);
}
Copy the codeDecode_in_io_thread_key – Whether to decode in THE I/O thread. The default is false to avoid processing business logic in the I/O thread. This is also recommended by Netty. Hence the asynchronous decoding process.
So look at the code that decodes the business object, remember where that was? DecodeableRpcResult#decode
DecodeableRpcResult
@Override
public Object decode(Channel channel, InputStream input) throws IOException {
ObjectInput in = CodecSupport.getSerialization(channel.getUrl(), serializationType)
.deserialize(channel.getUrl(), input);
byte flag = in.readByte();
switch (flag) {
case DubboCodec.RESPONSE_NULL_VALUE:
// ...
case DubboCodec.RESPONSE_VALUE_WITH_ATTACHMENTS:
handleValue(in);
handleAttachment(in);
break;
case DubboCodec.RESPONSE_WITH_EXCEPTION_WITH_ATTACHMENTS:
// ...
default:
throw new IOException("Unknown result flag, expect '0' '1' '2' '3' '4' '5', but received: " + flag);
}
// ...
return this;
}
private void handleValue(ObjectInput in) throws IOException {
try {
Type[] returnTypes;
if (invocation instanceof RpcInvocation) {
returnTypes = ((RpcInvocation) invocation).getReturnTypes();
} else {
returnTypes = RpcUtils.getReturnTypes(invocation);
}
Object value = null;
if (ArrayUtils.isEmpty(returnTypes)) {
// This almost never happens?
value = in.readObject();
} else if (returnTypes.length == 1) { value = in.readObject((Class<? >) returnTypes[0]);
} else{ value = in.readObject((Class<? >) returnTypes[0], returnTypes[1]);
}
setValue(value);
} catch(ClassNotFoundException e) { rethrow(e); }}Copy the codeNow that ObjectInput is here, what is the underlying serialization framework selection logic? How to stay consistent with the serialization framework on the consumer side?
Each a serialization framework has a id see org.apache.dubbo.com mon. Serialize. Constants;
When requested, the serialization framework is selected based on the Url information. The default is hessian2
Serialization framework identity is written into the protocol header during transmission. For details, see Exchange Dec# encodeRequest#218
When a request is received from the consumer side, the corresponding serialization framework is used based on this ID.
The actual holding object is Hessian2ObjectInput. Due to the complex deserialization logic of readObject, the process is as follows:
4. Common problems
Q1: the provider changed the classpath in the Facade, but the consumer deserialized without an error?
A: During deserialization, if the consumer cannot find the classpath returned by the provider, an exception will be caught and the local return type will prevail
Question 2: When encoding serialization, there is no reason to write the return value?
A: Because in Java, the return value is not one of the pieces of information that identifies a method
Question 3: When will A and B be inconsistent in the deserialization flowchart? Where is A’s information read from?
A: When the provider changes the classpath, A and B will appear different; The A information comes from the Invocation context stored in the Request object, which is the return type in the local JAR.
Q4: Does the consumer report an error when the provider adds or deletes return fields?
A: No, when deserializing, take the intersection of the two fields.
Q5: Does the consumer report an error when providing information about the parent class of the modified object?
A: No, the transmission carries only the field information of the parent class. When instantiated, the local class is instantiated, not associated with the parent classpath of the provider’s actual code.
Question 6: During deserialization, what happens if the returned object subclass and its parent class have fields of the same name, and the subclass has values, but the parent class has no values?
A: In dubo-3.0.x, the return field will be empty. The reason is that when the encoding side iterates over the set of fields (which may be encoded by the consumer side or the provider side), the field information of the parent class comes after the subclass. When the decoding side iterates on the field set, it gets the deserializer through the field key. In this case, the subclass has the same name as the parent class, so the first reflection sets the subclass value, and the second reflection sets the parent class value for overwriting.
In dubo-2.7.x, this problem has been resolved. The solution is also simpler, to reverse the field order via collections.reverse (fields) during encoding side transport.
JavaSerializer
public JavaSerializer(Class cl, ClassLoader loader) {
introspectWriteReplace(cl, loader);
// ...
List fields = new ArrayList();
fields.addAll(primitiveFields);
fields.addAll(compoundFields);
Collections.reverse(fields);
// ...
}
Copy the codeWrite at the end
The encoding and decoding process is complicated and obscure, and the data types are diverse. The author encountered and understood limited, with the most common, the simplest data type to comb the process of codec. Please forgive me for any mistakes or omissions.
Author: Sun Wen, Vivo Internet Server Team