Netty uses buffer design to improve transmission efficiency during data transmission. Although Java has provided the ByteBuffer class for use in NIO programming, the encoding method is relatively unfriendly and has some shortcomings. So the high-performance Netty framework implements a more powerful, improved ByteBuf, and its design concept is also a must.

ByteBuffer analysis

Before we look at ByteBuf, let’s talk briefly about the operations of the ByteBuffer class. For better understanding ByteBuf.

Read and write operations of ByteBuffer share a position pointer. The read and write process is analyzed through the following code cases:

// Allocate a buffer and specify the size
ByteBuffer buffer = ByteBuffer.allocate(100);
// Set the current maximum cache size limit
buffer.limit(15);
System.out.println(String.format("allocate: pos=%s lim=%s cap=%s", buffer.position(), buffer.limit(), buffer.capacity()));

String content = "Ytao Official Account";
// Write data to the buffer
buffer.put(content.getBytes());
System.out.println(String.format("put: pos=%s lim=%s cap=%s", buffer.position(), buffer.limit(), buffer.capacity()));
Copy the code

Three parameters of the buffer are printed, respectively:

  • Position Indicates the position of the read or write pointer
  • Limit Indicates the current cache size limit
  • Capacity Buffer size

Print result:

When we write the content, the read/write pointer value is 13, yTAO public number English characters of 1 byte, each Chinese characters of 4 bytes, just 13, less than the current buffer size of 15.

Next, read the yTAO data in the content:

buffer.flip();
System.out.println(String.format("flip: pos=%s lim=%s cap=%s", buffer.position(), buffer.limit(), buffer.capacity()));

byte[] readBytes = new byte[4];
buffer.get(readBytes);
System.out.println(String.format("get(4): pos=%s lim=%s cap=%s", buffer.position(), buffer.limit(), buffer.capacity()));

String readContent = new String(readBytes);
System.out.println("readContent:"+readContent);
Copy the code

To read the content, create a byte array to receive and specify the size of the received data.

After writing data to read content, you must actively call ByteBuffer#flip or ByteBuffer#clear.

ByteBuffer#flip will take the value of the pointer position after writing data as the current buffer size, and then zero the pointer position. Changes the buffer for writing data to the buffer for fetching data, that is, data is read from the first index of the newly written data as the starting index.

ByteBuffer#flip

ByteBuffer#clear resets limit to the default value, which is the same size as capacity.

Read the rest:

On the second read, buffer#remaining obtains bytes greater than or equal to the remaining content. This function is implemented as limit-position, so the current buffer area must be within this value.

readBytes = new byte[buffer.remaining()];
buffer.get(readBytes);
System.out.println(String.format("get(remaining): pos=%s lim=%s cap=%s", buffer.position(), buffer.limit(), buffer.capacity()));
Copy the code

Print result:

During the above operations, the index changes as shown in the figure:

ByteBuf Read/write operations

ByteBuf has separate read/write Pointers, namely buf#readerIndex and buf#writerIndex. The current buffer size is buf#capacity.

Here the buffer is divided into three regions by two pointer indexes and capacity:

  • 0 -> readerIndex indicates the read buffer area, which can be reused to save memory. The value of readerIndex is greater than or equal to 0
  • ReaderIndex -> writerIndex indicates the buffer area that can be read. The value of writerIndex is greater than or equal to readerIndex
  • WriterIndex -> Capacity indicates the writerbuffer area. Capacity is greater than or equal to writerIndex

As shown below:

Allocate buffer

ByteBuf allocates a buffer, given only an initial value. The default is 256. Unlike ByteBuffer, the initial value is not the maximum value. The maximum value of ByteBuf is integer.max_value

ByteBuf buf = Unpooled.buffer(13);
System.out.println(String.format("init: ridx=%s widx=%s cap=%s", buf.readerIndex(), buf.writerIndex(), buf.capacity()));
Copy the code

Print result:

The write operation

ByteBuf writes are similar to ByteBuffer except that the write pointer is recorded separately. ByteBuf writes support multiple types and have the following apis:

Write byte array type:

String content = "Ytao Official Account";
buf.writeBytes(content.getBytes());
System.out.println(String.format("write: ridx=%s widx=%s cap=%s", buf.readerIndex(), buf.writerIndex(), buf.capacity()));
Copy the code

Print result:

Index diagram:

A read operation

Similarly, ByteBuf writes are similar to ByteBuffer except that the write pointer is recorded separately. ByteBuf reads support multiple types and have the following apis:

Read four bytes from the current readerIndex position:

byte[] dst = new byte[4];
buf.readBytes(dst);
System.out.println(new String(dst));
System.out.println(String.format("read(4): ridx=%s widx=%s cap=%s", buf.readerIndex(), buf.writerIndex(), buf.capacity()));
Copy the code

Print result:

Index diagram:

ByteBuf Dynamic capacity expansion

Using the ByteBuffer example above, add [ytao public id ytao public id] to the buffer to make it write more than the limit value.

ByteBuffer buffer = ByteBuffer.allocate(100);
buffer.limit(15);

String content = "Ytao Official Account yTAO Official Account";
buffer.put(content.getBytes());
Copy the code

Abnormal running result:

Buffer overflow exceptions occur when the content byte size exceeds the limit value, so we have to check whether the buffer size is sufficient before writing data, which is not a good coding experience.

Add the same content using ByteBuf, given the same initial container size.

ByteBuf buf = Unpooled.buffer(15);
String content = "Ytao Official Account yTAO Official Account";
buf.writeBytes(content.getBytes());
System.out.println(String.format("write: ridx=%s widx=%s cap=%s", buf.readerIndex(), buf.writerIndex(), buf.capacity()));
Copy the code

Print run results:

According to the printed information above, we can see that cap has changed from 15 to 64. When the container size is insufficient, we need to expand the capacity. Next, we will analyze how to do in the expansion process. Go to writeBytes:

Check write length:

In the writable area check:

  • If nothing is written, an invalid argument exception is thrown.
  • If the size of the write is less than or equal to the size of the writable area, returns the value ofwritableBytes()The function is the size of the writable regioncapacity - writerIndex
  • If the write size is greater than the maximum writable region size, an out-of-index exception is thrown.
  • All that remains is that if the size of the write is greater than the writable region and smaller than the maximum region size, a new buffer region is allocated.

In the insufficient capacity, redistribute the buffer inside, with 4M as the valve:

  • If the backlog is just 4M, a 4M buffer is allocated.
  • If the content to be written exceeds this valve and the sum of the valve values is not greater than the maximum capacity value, a buffer (valve value + content size value) is allocated; If this valve is exceeded and the sum of the valve values is greater than the maximum capacity value, a maximum capacity buffer is allocated.
  • If the backlog does not exceed the threshold and is greater than 64, the buffer size to be allocated is doubled by 64 until it is equal to or greater than the backlog.
  • If the buffer to be written does not exceed the valve value and does not exceed 64, the buffer to be allocated is returned with a size of 64.

The last

Netty Buffer implementation, eight basic types, except for the Boolean type, the other seven have their own corresponding Buffer, but in practice, we try to use ByteBuf, it can be compatible with any type. ByteBuf is the most basic and important member of Netty. To better understand and use Netty, you must first understand ByteBuf.


Personal blog: Ytao.top

Pay attention to the public number [Ytao], more original good articles