What is 1.Net TY?

Netty is an asynchronous event-driven network application framework for rapid development of maintainable high-performance protocol servers and clients. Netty is based on NIO and encapsulates THE NIO of the JDK, making it more flexible to use.

What are the features of 2.Net TY?

  • High concurrency: Netty is a network communication framework based on Nonblocking IO (NIO), compared to BIO (Blocking I/O), its concurrency performance is greatly improved.
  • Fast transmission: Netty’s transmission relies on the zero-copy feature to minimize unnecessary memory copies and achieve more efficient transmission.
  • Encapsulate: Netty encapsulates many of the details of NIO operations and provides an easy-to-use invocation interface.

3. What are the advantages of.NET TY?

  • Simple to use: Encapsulates many details of NIO, making it easier to use.
  • Powerful: preset a variety of encoding and decoding functions, support a variety of mainstream protocols.
  • Strong customization capability: The communication framework can be flexibly extended through ChannelHandler.
  • High performance: Compared with other mainstream NIO frameworks in the industry, Netty has the best overall performance.
  • Stability: Netty fixed all NIO bugs it had found, allowing developers to focus on the business itself.
  • Active community: Netty is an active open source project with short iteration cycles and fast bug fixes.

What are the application scenarios of 4.Net TY?

Typical applications include Dubbo, alibaba’s distributed services framework, which uses Netty by default as the basic communication component, and RocketMQ, which also uses Netty as the basic communication component.

5.What are the high-performance aspects of NET TY?

  • IO thread model: synchronously non-blocking, doing more with the fewest resources.
  • Zero memory copy: Minimizes unnecessary memory copy to achieve more efficient transmission.
  • Memory pool design: The allocated memory can be reused, mainly refers to the direct memory. The internal implementation manages memory allocation with a binary lookup tree.
  • Serialization handles reads and writes: Avoids the performance overhead of using locks.
  • High-performance serialization protocol: Supports high-performance serialization protocols such as Protobuf.

6. What are the differences between BIO, NIO and AIO?

BIO: One thread for each connection. When a client requests a connection, the server starts a thread to process it. Threads are expensive. Pseudo asynchronous IO: Request connections are put into a thread pool, one-to-many, but threads are still a valuable resource.

NIO: One thread per request, but all connection requests sent by the client are registered with the multiplexer. The multiplexer polls the connection for I/O requests and starts a thread for processing.

AIO: a valid request to a thread. The CLIENT’s I/O request is completed by the OS and then notified the server application to start the thread for processing.

BIO is flow-oriented, NIO is buffer-oriented; The BIO streams are blocked. NIO is non-blocking; BIO’s Stream is one-way, while NIO’s channel is bidirectional.

NIO features: event-driven model, single-thread processing multi-task, non-blocking I/O, I/O read and write no longer blocks, but returns 0, block-based transmission than stream-based transmission more efficient, more advanced IO functions zero-copy, IO multiplexing greatly improves the scalability and practicality of Java network applications. Based on the Reactor thread model.

In the Reactor model, the event distributor waits for an event or a state that can be applied to an action. The event distributor passes the event to a pre-registered event handler or callback that does the actual reading and writing. For example, read is implemented in Reactor: register read ready event, corresponding event handler, event distributor waits for event and event arrival, activate distributor, distributor calls handler corresponding to event to complete actual read operation, process read data, register new event, and then return control.

7. What is NIO made of?

Buffer: Interacts with a Channel. Data is read from a Channel into the Buffer and written from the Buffer into a Channel

Flip method: Reverse this buffer, give position to limit, and then set position to 0 to switch read/write mode

Clear method: Clear this buffer, set position to 0, and give the value of Capacity to limit.

Rewind method: rewind this buffer, setting position to 0

DirectByteBuffer reduces one copy from system space to user space. However, buffers are more expensive to create and destroy and are out of control, and memory pools are often used to improve performance. Direct buffers are primarily allocated to large, persistent buffers that are vulnerable to native I/O operations on the underlying system. For small and medium applications with small data volumes, heapBuffer can be used to be managed by the JVM.

Channel: indicates the bidirectional connection between the I/O source and the target. However, it cannot directly access data and can only interact with the Buffer. FileChannel’s read and write methods both cause data to be copied twice!

The Selector method allows a single thread to manage multiple channels, the open method creates the Selector, and the Register method registers the Channel with the multiplexer, listening for the types of events: read, write, connect, and Accept. The registration event generates a SelectionKey, which represents the registration relationship between the SelectableChannel and the Selector, and the wakeup method, which immediately returns the first selection that has not yet been returned

A new channel or event is registered. The channel is closed and the registration is cancelled. Higher priority events (such as timer events) are triggered and you want to process them in a timely manner.

The implementation class of Selector in Linux is EPollSelectorImpl, which delegates to the EPollArrayWrapper implementation. The three native methods encapsulate epoll, while the EPollSelectorImpl. ImplRegister method, Events are registered with the epoll instance by calling epoll_ctl. The mapping between the registered file descriptor (FD) and the SelectionKey is also added to the fdToKey. This map maintains the mapping between the file descriptor and the SelectionKey.

The fdToKey can sometimes become very large because of the number of channels registered with the Selector (millions of connections); Expired or invalid channels are not closed in time. The fdToKey is always read serially, and reading is done in the SELECT method, which is non-thread-safe.

Pipe: one-way data connection between two threads. Data will be written to the sink channel and read from the source channel

Selector. Open () : opens a Selector; Serversocketchannel.open () : Create a Channel on the server; Bind () : binds to a port. Configure the non-blocking mode. Register () : registers channels and concerned events to Selector; Select () polls to get the events that are already in place

8.Net TY thread model?

Netty receives and processes user requests based on the Reactor model and multiplexer, and internally implements two thread pools, boss thread pool and Work thread pool. The boss thread pool is responsible for processing the accept event of the request. When receiving the accept event request, Encapsulate the corresponding socket into a NioSocketChannel and hand it to the work thread pool, which takes care of the requested read and write events and handles them by the corresponding Handler.

Single-threaded model: All I/O operations are done by one thread, meaning that multiplexing, event distribution, and processing are done by one Reactor thread. You need to receive a client connection request, initiate a connection to the server, and send/read a request or reply/response message. A NIO thread processing hundreds of links at the same time, performance cannot support, slow, if the thread into an infinite loop, the whole program is unavailable, not suitable for high load, large concurrency application scenarios.

Multithreaded model: there is a NIO thread (Acceptor) that only listens to the server and receives TCP connection requests from the client. NIO thread pools are responsible for the operations of network IO, that is, reading, decoding, encoding, and sending messages; One NIO thread can process N links at the same time, but one link corresponds to only one NIO thread to prevent concurrent operations. However, an Acceptor thread can suffer from performance problems when millions of clients are connected or security authentication is required.

Master-slave multithreading model: Acceptor threads are used to bind listener ports, receive client connections, remove socketChannels from the multiplexer of the Reactor thread from the main Reactor thread, and re-register them with the Sub thread for I/O operations. So that mainReactor is only responsible for access authentication, handshake and other operations;

9. Causes and solutions of TCP packet sticking/unpacking?

TCP processes data in the form of streams. A complete packet may be divided into multiple packets and sent by TCP, or a small packet may be encapsulated into a large packet and sent.

TCP sticky packet/subcontract causes:

If the bytes written by the application program are larger than the size of the socket sending buffer, packet unpacking will occur. If the data written by the application program is smaller than the size of the socket sending buffer, the network adapter will send the data written by the application for several times to the network, which will cause packet sticking.

When the TCP packet length -TCP header length is greater than MSS, the payload (net load) of the Ethernet frame is greater than the MTU (1500 bytes) for IP fragmentation.

The solution

Message length: The FixedLengthFrameDecoder class

Add special character split at the end of package:

  • Line separator class: LineBasedFrameDecoder
  • Or custom delimiter: DelimiterBasedFrameDecoder

The message into the message header and the message body: LengthFieldBasedFrameDecoder class. It can be divided into unpacking and sticking with a head, unpacking and sticking with a head before the length field, and unpacking and sticking with multiple extended heads.

10. What is Netty zero copy?

Netty zero copy has three main aspects:

  • Netty receives and sends bytebuffers using DIRECT BUFFERS, which use out-of-heap DIRECT memory for Socket reading and writing without the need for secondary copy of byte BUFFERS. If traditional HEAP BUFFERS are used for Socket reads and writes, the JVM copies the HEAP Buffer into direct memory before writing it to the Socket. The message is sent with an extra memory copy of the buffer compared to direct out-of-heap memory.
  • Netty provides the combined Buffer object, which can aggregate multiple ByteBuffer objects. Users can operate the combined Buffer as easily as operating a single Buffer. The traditional memory copy method of combining several small buffers into one large Buffer is avoided.
  • Netty adopts the transferTo method to transfer files, which can directly send the data in the file buffer to the target Channel, avoiding the memory copy problem caused by the traditional write method.

What are the important components in 11.Net TY?

  • Channel: Netty network operation abstract class, which includes basic I/O operations, such as bind, connect, read, write, etc.
  • EventLoop: Works with a Channel to handle I/O operations and handle events that occur during the lifetime of the connection.
  • ChannelFuture: All I/O operations in the Netty framework are asynchronous, so we need the addListener() of ChannelFuture to register a ChannelFutureListener to listen for events that will automatically return results when the operation succeeds or fails.
  • ChannelHandler: Acts as a logical container for all the processing of inbound and outbound data. ChannelHandler is used to handle a wide range of events, such as connections, data receiving, exceptions, data conversion, etc.
  • ChannelPipeline: Provides a container for the ChannelHandler chain. When a channel is created, it is automatically assigned to its own Channel pipeline. The association is permanent.

12.How many Ways can NET TY Send Messages?

Netty can send messages in two ways:

  • Writing directly to a Channel, the message moves from the middle end of the Channel pipeline;
  • Writing to the ChannelHandlerContext bound to ChannelHandler, the message is moved from the next ChannelHandler in the ChannelPipeline.

How many threads does Netty have by default? When does it start?

Netty defaults to twice the number of CPU processors and starts when bind is done.

14. What serialization protocols do you know?

Serialization (encoding) is to serialize an object into binary form (byte array), mainly used for network transmission, data persistence, etc. Deserialization (decoding) is the restoration of byte arrays read from the network, disk, etc., to the original object, mainly used for decoding the network transfer object, in order to complete the remote call.

The key factors affecting serialization performance are the size of serialization stream (network bandwidth consumption) and serialization performance (CPU resource consumption). Whether support cross-language (heterogeneous system docking and development language switch).

Java provides serialization by default: it cannot cross languages, the serialized stream is too large, and the serialization performance is poor

XML, advantages: human-machine readable, element or feature names can be specified. Disadvantages: Serialized data only contains the data itself and the structure of the class, excluding type identification and assembly information; Only public properties and fields can be serialized; Cannot serialize methods; The file is huge, the file format is complex, and the transmission takes up bandwidth. Application scenario: Stores data as a configuration file and converts data in real time.

JSON, a lightweight data exchange format, has the following advantages: high compatibility, simple data format, easy to read and write, small serialized data, good scalability, good compatibility, compared with XML, its protocol is relatively simple, fast parsing. Disadvantages: Data is less descriptive than XML, not suitable for ms performance requirements, and high overhead of extra space. Application scenario (alternative to XML) : cross-firewall access, high adjustability requirements, Ajax requests based on Web Browser, relatively small amount of data transfer, and relatively low real-time requirements (such as second level).

Fastjson, which uses a “assume ordered fast matching” algorithm. Advantages: Easy to use interface, currently the fastest JSON library in the Java language. Disadvantages: too much focus on speed, deviation from “standards” and functionality, poor code quality, incomplete documentation. Application scenario: Protocol interaction, Web output, and Android client

Thrift is not only a serialization protocol, but also an RPC framework. Advantages: small size after serialization, fast speed, supports multiple languages and rich data types, has strong compatibility for the addition and deletion of data fields, supports binary compression coding. Disadvantages: few users, unsafe and unreadable when accessed across firewalls, relatively difficult to debug code, cannot be used with other transport protocols (such as HTTP), cannot read and write data directly to the persistence layer, that is, not suitable for persistent serialization protocol. Application scenario: RPC solution for distributed systems

Avro, a subproject of Hadoop, addresses JSON’s verbosity and lack of IDL. Advantages: support rich data types, simple dynamic language combination, self-describing properties, improved data parsing speed, fast and compressible binary data form, remote procedure call RPC, support cross-programming language implementation. Disadvantages: Not intuitive for users accustomed to statically typed languages. Application scenario: Hive, Pig, and MapReduce persistent data formats in Hadoop.

For Protobuf, data structures are described in a. Proto file. Code generation tools can generate POJO objects corresponding to data structures and methods and properties related to Protobuf. Advantages: Small code stream after serialization, high performance, structured data storage format (SUCH as XML JSON), forward compatibility of protocols by identifying the sequence of fields, and easier management and maintenance of structured documents. Disadvantages: rely on tools to generate code, relatively few languages are supported, only Java, C++ and python are officially supported. Application scenario: RPC calls with high performance requirements, good cross-firewall access attributes, and persistence of objects at the application layer

other

Jboss Marshaling can serialize Java classes directly, No need for real Java.io.Serializable interface Message Pack an efficient binary serialization format Hessian uses the binary protocol lightweight remoting onHTTP tool Kryo based on the Protobuf protocol, Only support Java language, need to register, and then serialize (Output), deserialize (Input)

15. How to select a serialization protocol?

The scenario

For inter-company system calls, xmL-based SOAP is a worthwhile solution if the performance requirements are above 100ms for services. Ajax based on Web Browser, and communication between Mobile APP and server, JSON protocol is the first choice. JSON is also a good choice for scenarios where performance is not very high, where dynamic typed languages are dominant, or where data transfer loads are small. Using JSON or XML can greatly improve debugging efficiency and reduce system development costs in the scenario where the debugging environment is harsh. Protobuf, Thrift, and Avro have some competition in scenarios where performance and simplicity are extremely important. Protobuf and Avro are the first choice for persistence scenarios with T-level data. If persistent data is stored in a Hadoop subproject, Avro is a better choice.

Protobuf is more suitable for statically typed language engineers for non-Hadoop persistence projects. Because Avro’s design philosophy favors dynamically typed languages, Avro is a better choice for dynamic language-dominated applications. Thrift is a good choice if you need to provide a complete RPC solution. Protobuf can be preferred if different transport layer protocols need to be supported after serialization, or if high performance scenarios require cross-firewall access. The protobuf data types are bool, double, float, int32, int64, string, bytes, enum, and message. For a protobuf, the qualifier is required: the value must not be empty. For a optional field, the value can be assigned to a specific field or not. Only one value from the specified set of constants can be used as its value.

Basic rules for protobuf: each message must have at least one required field and zero or more Optional fields; A repeated field can contain zero or more data. Identification numbers within [1,15] will occupy one byte during encoding (commonly used), and identification numbers within [16,2047] will occupy two bytes. The identification numbers must not be repeated, and the message type can also be nested at any level, and the nested message type can be used instead of group.

The protobuf message upgrade principle: do not change the numeric identifier of any existing field; Existing required fields cannot be removed. Optional and repeated fields can be removed, but labels cannot be reused. The newly added field must be optional or repeated. Older programs cannot read or write fields of the new Required qualifier.

The compiler generates a. Java file for each message type, as well as a special Builder class that creates the message-class interface. Such as: UserProto. User. Builder Builder = UserProto. User. NewBuilder (); Builder. The build ();

The use of Netty: ProtobufVarint32FrameDecoder is used to deal with half a pack message decoding classes; ProtobufDecoder (UserProto. User. GetDefaultInstance ()). This is to create UserProto Java file decoding classes; ProtobufVarint32LengthFieldPrepender messages about the protobuf agreement on the head with a length of 32 plastic field, used to mark the length of the message class; ProtobufEncoder is the code class

Convert StringBuilder to type ByteBuf: copiedBuffer() method

16.What Heartbeat Types does NET TY Support?

  • ReaderIdleTime: indicates the read timeout period (the test end does not receive any message from the tested end within a specified period of time).
  • WriterIdleTime: indicates the write timeout period (the test end sends a message to the tested end within a certain period of time).
  • AllIdleTime: Timeout for all types.

17.Net TY and Tomcat?

  • Different functions: Tomcat is a Servlet container that can be viewed as a Web server, while Netty is an asynchronous event-driven network application framework and tools for simplifying network programming, such as TCP and UDP socket servers.
  • Different agreements: Tomcat is a Web server based on HTTP protocol, and Netty can customize various protocols programmatically, because Netty itself can encode/decode byte streams, all of which Netty can implement. HTTP server, FTP server, UDP server, RPC server, WebSocket server, Redis Proxy server, MySQL Proxy server and so on.

18. NIOEventLoopGroup source?

NioEventLoopGroup (actually MultithreadEventExecutorGroup) internal maintenance a type to EventExecutor children [], the default size is the number of processor cores * 2, thus form a thread pool, NioEventLoopGroup overloads the newChild method when initializing EventExecutor, so the actual type of the children element is NioEventLoop.

Thread startup called SingleThreadEventExecutor constructor, perform NioEventLoop run method of a class, first call hasTasks () method to judge whether the current taskQueue elements. If there are elements in the taskQueue, the selectNow() method is executed, and eventually selector. SelectNow () is executed, which returns immediately. If the taskQueue has no elements, perform the select(oldWakenUp) method

Select (oldWakenUp) is used to record the number of times the selectCnt method has been executed and to identify whether the selectNow() method has been executed. Select (timeoutMillis) is repeatedly executed, and the variable selectCnt becomes larger and larger. When the selectCnt reaches the threshold (512 by default), the rebuildSelector method is executed to rebuild the selector. Fixed the CPU usage bug of 100%.

The rebuildSelector method starts by creating a new selector using the openSelector method. And then cancel the selectionKey of the old selector. Finally, the old selector’s channel is re-registered with the new selector. After rebuild, you need to re-execute the selectNow method to check whether the selectionKey is ready.

The processSelectedKeys method (which handles the I/O task) is then called when selectedKeys! = null, call processSelectedKeysOptimized method, Iterating selectedKeys gets the selectkey of the ready IO event stored in the array selectedKeys, then calls processSelectedKey for each event to handle it, ProcessSelectedKey handles OP_READ; OP_WRITE; OP_CONNECT events.

The last call runAllTasks method (IO), the method will first call fetchFromScheduledTaskQueue method, has more than delay the time in scheduledTaskQueue task is waiting to be performed in moved to taskQueue, If the execution time of 64 tasks exceeds the preset execution time, stop the execution of non-I/O tasks to prevent I/O task execution from being affected by too many non-I/O tasks.

Each NioEventLoop corresponds to a thread and a Selector. The NioServerSocketChannel will actively register with a Selector of a NioEventLoop, which is responsible for event polling.

Outbound events are request events. The initiator is Channel, and the handler is unsafe. Outbound events are notified through an unsafe event, and the propagation direction is from tail to head. The initiator of an Inbound event is an unsafe event. The handler of an event is a Channel, which is a notification event. The propagation direction is from beginning to end.

For the memory management mechanism, a large Chunk of memory Arena will be pre-applied for. Arena consists of many chunks, and each Chunk consists of 2048 pages by default. Chunk organizes pages in an AVL tree, with each leaf node representing a Page and the middle node representing an area of memory, and the node itself records its offset address across the Arena. When the region is allocated, the marker bits on the intermediate node are marked, indicating that all nodes below the intermediate node have been allocated. The poolChunkList allocates memory larger than 8K, and PoolSubpage allocates memory smaller than 8K, which splits a page into multiple segments for memory allocation.

ByteBuf supports automatic capacity expansion (4M), ensures that the PUT method will not throw exceptions, and realizes zero-copy through the built-in compound buffer type. There is no need to call flip() to switch the read/write mode; read and write indexes are separate; Methods chain; Based on reference counting AtomicIntegerFieldUpdater for memory recovery; PooledByteBuf uses a binary tree to implement a memory pool that centrally manages the allocation and release of memory without creating a new buffer object for each use. UnpooledHeapByteBuf creates a new buffer object each time.

Introduction of Netty

Netty is an asynchronous event-driven network application framework for rapid development of maintainable high-performance protocol servers and clients.

JDK native NIO program issues

The JDK native also has a set of web application apis, but there are a number of problems, mainly as follows:

  • NIO class libraries and apis are cumbersome to use, you need to master Selector, ServerSocketChannel, SocketChannel, ByteBuffer, etc
  • Additional skills are required, such as familiarity with Java multithreaded programming. Since NIO programming involves the Reactor model, you must be familiar with multithreaded and network programming in order to write high quality NIO programs
  • The reliability of the ability to complement, the workload and difficulty of development are very large. For example, the client is faced with disconnection and reconnection, network intermittent disconnection, half-packet read and write, failure cache, network congestion and abnormal code stream processing, etc. NIO programming is characterized by relatively easy function development, but the workload and difficulty of reliability capability complement are very large
  • JDK NIO bugs, such as the infamous epoll BUG, cause Selector polling to be empty, eventually causing the CPU to go 100%. It was claimed that the issue was fixed in update18 in JDK1.6, but the issue still exists in JDK1.7, although the bug is less likely to occur and not completely resolved

The characteristics of Netty

Netty on the JDK built-in NIO API encapsulation, to solve the above problems, the main features are:

  • Unified API elegently designed for all transport types – Blocking and non-blocking sockets based on flexible and extensible event model with clear separation of concerns highly customizable threading model – single-threaded, one or more thread pools true connectionless datagram Socket support (since 3.1)
  • JDK 5 (Netty 3.x) or 6 (Netty 4.x) will suffice
  • High performance Higher throughput and lower latency Reduce resource consumption and minimize unnecessary memory replication
  • Secure complete SSL/TLS and StartTLS support
  • Active community, constant updates active community, short version iteration cycle, found bugs can be fixed in time, at the same time, more new features will be added

Common usage scenarios of Netty

Common Netty scenarios are as follows:

  • In the Internet industry, remote service invocation is required between nodes in distributed systems, and high-performance RPC frameworks are essential. Netty, as an asynchronous and high-tech communication framework, is often used by these RPC frameworks as a basic communication component. Typical applications are as follows: Ali distributed service framework Dubbo RPC framework uses Dubbo protocol for inter-node communication. Dubbo protocol uses Netty as the basic communication component by default, which is used to realize internal communication between process nodes.
  • Java language has been more and more widely used in the game industry, whether it is mobile game server or large online game. As a high-performance basic communication component, Netty itself provides TCP/UDP and HTTP protocol stacks. It is very convenient to customize and develop private protocol stack, account login server, map server can facilitate high-performance communication through Netty
  • The RPC framework of Avro, a classic Hadoop high-performance communication and serialization component in the field of big data, uses Netty for cross-border point communication by default, and its Netty Service is implemented based on the secondary encapsulation of Netty framework

Interested readers can find out which open source projects currently use Netty: Related Projects

Netty high performance design

As an asynchronous event-driven network, Netty’s high performance mainly comes from its I/O model, which determines how to send and receive data, and its threading model, which determines how to process data

I/O model

What channels are used to send data to each other, BIO, NIO, or AIO, and the I/O model largely determines the performance of the framework

Blocking I/O

Traditional blocking I/O(BIO) can be represented as follows:

The characteristics of

  • Each request requires a separate thread to complete data read, business processing, and data write operations

The problem

  • When the number of concurrent connections is large, a large number of threads need to be created to process connections, occupying large system resources
  • After a connection is established, if the current thread has no data to read temporarily, the thread blocks on the read operation, resulting in a waste of thread resources

I/O multiplexing model

In the I/O reuse model, use the select, this function will make process jams, but and blocking I/O are different, the two function can block multiple I/O operations at the same time, but also at the same time for multiple read operations, multiple write I/O functions for testing, until there is data can be read or write, truly call I/O operations function

The key to Netty’s non-blocking I/O implementation is based on the I/O multiplexing model, which is represented by a Selector object:

Netty’s IO thread NioEventLoop can process hundreds or thousands of client connections simultaneously due to its aggregation of multiplexer selectors. When a thread reads or writes data from a client Socket channel, it can perform other tasks if no data is available. Threads typically spend idle time of non-blocking IO performing IO operations on other channels, so a single thread can manage multiple input and output channels.

Because the read and write operations are non-blocking, this can fully improve the running efficiency of THE IO thread, avoiding the thread suspension caused by frequent I/O blocking. One I/O thread can concurrently process N client connections and read and write operations, which fundamentally solves the traditional synchronous blocking I/O one connection one thread model. The performance, flexibility and reliability of the architecture have been greatly improved.

Based on the buffer

Traditional I/O is byte Stream or character Stream oriented and reads one or more bytes sequentially from a Stream in a streaming manner, so the position of the read pointer cannot be arbitrarily changed.

In NIO, the traditional I/O streams were abandoned and the concepts of channels and buffers were introduced. In NIO, data can only be read from a Channel to a Buffer or written from a Buffer to a Channel.

Unlike the sequential operations of traditional IO, buffer-based operations can read data at random in NIO

Threading model

How to read datagrams? The thread in which the codec is carried out after the read, how the message is distributed after the codec, and the thread model have great influence on the performance.

Event-driven model

In general, there are two approaches to designing an event-handling model

  • In polling mode, the thread continuously polls the source of the related event to see if an event has occurred. If an event has occurred, the event processing logic will be invoked.
  • In event-driven mode, the main thread puts the event into the event queue, and the other thread continuously circulates and consumes the events in the event list, invoking the corresponding processing logic of the event to process the event. The event-driven approach, also known as the message notification approach, is the idea behind the observer approach in design patterns.

Take GUI logic processing as an example to illustrate the difference between the two logics:

  • Polling mode Threads continually poll for button click events and invoke processing logic if they do
  • Event-driven event occurrence click event puts the event in the event queue, in the event list consumed by another thread, invokes the relevant event handling logic based on the event type

Here is O ‘Reilly’s illustration of the event-driven model

It consists of four basic components:

  • Event Queue: An entry to receive events and store events to be processed
  • Event Mediators: Distribute different events to different business logic units
  • Event Channels: Communication channels between dispensers and processors
  • Event Processor: Implements the service logic. After the processing is complete, an event is emitted to trigger the next operation

It can be seen that compared with the traditional polling mode, event-driven has the following advantages:

  • Good scalability, distributed asynchronous architecture, high decoupling between event processors, can easily extend the event processing logic
  • High performance, based on queue temporary event, can facilitate parallel asynchronous processing of events

Reactor thread model

Reactor means Reactor model, which refers to the event-driven processing of service requests through one or more inputs to the service processor simultaneously. The server program processes the incoming multiple requests and synchronously dispatches them to the corresponding processing thread. The Reactor model is also called the Dispatcher model, which means that I/O is multiplexed with unified monitoring events and dispatched to a process after receiving the events. It is one of the necessary techniques for programming high performance network servers.

There are two key components in the Reactor model:

  • A Reactor runs in a separate thread, listening for and distributing events to the appropriate handlers to react to IO events. It acts like a corporate telephone operator, answering calls from customers and redirecting the line to the appropriate contact
  • Handlers perform the actual event that the I/O event is to accomplish, similar to the actual official in the company that the customer wants to talk to. Reactor responds to I/O events by scheduling appropriate handlers that perform non-blocking actions

Depending on the number of reactors and the number of Hanndler threads, there are three variants of the Reactor model

  • Single Reactor Single thread
  • Single Reactor multithreading
  • Reactor is multithreaded

A Reactor is a code that executes while (true) {selector. Select (); … } a cyclic thread, which produces an endless stream of new events, is aptly called a reactor.

There is no longer a specific comparison of Reactor characteristics, advantages and disadvantages. Readers who are interested may refer to my previous article: Understanding High-performance Network Models.

Netty threading model

Netty mainly makes some modifications to the multithreaded model of Master/slave Reactors (as shown in the following figure). There are multiple principal/slave Reactors and Subreactors in the multithreaded model.

  • MainReactor takes care of client connection requests and passes them on to SubReactor
  • The SubReactor is responsible for THE I/O requests of the corresponding channel
  • Tasks that are not IO requests (specific logical processing) are written directly to the queue and wait for worker Threads to process them

Scalable IO in Java: Scalable multithreading model for a Master-slave Reactor

It is particularly noted that although the thread model of Netty is based on master/slave Reactor multithreading and borrows the structure of MainReactor and SubReactor, in practice, SubReactor and Worker thread are in the same thread pool:

EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
ServerBootstrap server = new ServerBootstrap();
server.group(bossGroup, workerGroup)
 .channel(NioServerSocketChannel.class)

123456
Copy the code

The bossGroup and workerGroup in the code above are the two objects passed in the Bootstrap constructor, and both groups are thread pools

  • The bossGroup thread pool simply binds a port to one of the threads as a MainReactor that processes accept events for the port, with one boss thread for each port
  • The workerGroup thread pool is fully utilized by each SubReactor and worker thread

Asynchronous processing

Asynchrony is the opposite of synchronization. When an asynchronous procedure call is made, the caller does not get the result immediately. The part that actually handles the call notifies the caller through status, notifications, and callbacks after completion.

In Netty, I/O operations are asynchronous. Bind, write, connect and other operations simply return a ChannelFuture. The caller does not get the result immediately. Users can obtain THE I/O operation results proactively or through notification mechanisms.

When the future object is created, it is in an incomplete state. The caller can use the returned ChannelFuture to obtain the state of the operation and register the listener function to perform the completed operation. Common operations include:

  • The isDone method is used to check whether the current operation is complete
  • The isSuccess method is used to determine whether the completed current operation was successful
  • Use the getCause method to get the cause of the failure of the current completed operation
  • The isCancelled method is used to determine whether the completed current operation has been cancelled
  • The addListener method is used to register listeners. When the operation completes (isDone returns complete), the specified listener is notified. If the Future object is completed, the specified listener is understood to be notified

For example, in the following code, the binding port is an asynchronous operation. When the binding operation is finished, the corresponding listener processing logic will be invoked

serverBootstrap.bind(port).addListener(future -> { if (future.isSuccess()) { System.out.println(new Date() + ": Port [" + port + "] bound successfully! ); } else {system.err. Println (" port [" + port + "] binding failed!" ); }}); 12345678Copy the code

Compared to traditional blocking I/O, the thread blocks after an I/O operation until the operation is complete. The benefits of asynchronous processing are that threads do not block and can execute other programs during I/O operations, resulting in greater stability and throughput in high concurrency situations.

Netty architecture design

After introducing some theories about Netty, the following describes the Netty architecture design in terms of functions, features, modules, components, and operating processes

features

  • Transport services support BIO and NIO
  • Container integration supports OSGI, JBossMC, Spring, Guice containers
  • Protocol Support COMMON protocols such as HTTP, Protobuf, binary, text, and WebSocket. It also supports implementing custom protocols by implementing encoding and decoding logic
  • Core Extensible event model, common communication API, zero-copy ByteBuf buffer object

Module components

The Bootstrap, ServerBootstrap

Bootstrap means Bootstrap. A Netty application usually starts with a Bootstrap to configure the entire Netty program and connect various components. In Netty, the Bootstrap class is the Bootstrap class for the client program, and the ServerBootstrap class is the Bootstrap class for the server.

The Future, ChannelFuture

As mentioned earlier, all IO operations in Netty are asynchronous. It is not immediately clear whether the message is correctly processed or not, but you can wait for it to complete or register a listener. This is implemented through Future and ChannelFutures. A listener automatically triggers a registered listener event when the operation succeeds or fails.

Channel

Netty A component of network communication that performs network I/O operations. Channel provides users with:

  • The status of the channel currently connected to the network (for example, open? Is it connected?
  • Configuration parameters for the network connection (such as receive buffer size)
  • Provides asynchronous network I/O operations (such as establishing connections, reading and writing, binding ports). Asynchronous calls mean that any I/O calls are returned immediately and there is no guarantee that the requested I/O operation has been completed at the end of the call. The call immediately returns an instance of ChannelFuture, and by registering listeners on ChannelFuture, the caller can be notified of a successful, failed, or canceled I/O callback.
  • Supports associated I/O operations with corresponding handlers

Different types of channels correspond to connections of different protocols and blocking types. The following are some common Channel types

  • NioSocketChannel, asynchronous client TCP Socket connection
  • NioServerSocketChannel, an asynchronous server TCP Socket connection
  • NioDatagramChannel, asynchronous UDP connection
  • NioSctpChannel, asynchronous client Sctp connection
  • NioSctpServerChannel, asynchronous Sctp server side connection These channels cover UDP and TCP network IO and file IO.

Selector

Netty implements I/O multiplexing based on a Selector object. With a Selector, a thread can listen for multiple connected Channel events. After registering a Channel in a Selector, The Selector mechanism can automatically and continuously query the registered channels for ready I/O events (such as readable, writable, network connection completed, etc.), making it easy for the program to efficiently manage multiple channels with a single thread.

NioEventLoop

NioEventLoop maintains a thread and task queue, which supports asynchronous submission of tasks. When the thread is started, NioEventLoop’s RUN method is called to execute I/O tasks and non-I /O tasks:

  • I/O tasks are the ready events in selectionKey, such as Accept, Connect, read, write, and so on, triggered by processSelectedKeys.
  • Non-io tasks Tasks that are added to a taskQueue, such as register0 and bind0, are triggered by the runAllTasks method.

The execution time ratio of the two tasks is controlled by ioRatio. The default value is 50, indicating that the time allowed for non-I/O tasks is the same as that for I/O tasks.

NioEventLoopGroup

NioEventLoopGroup, which manages the life cycle of eventLoop, can be understood as a thread pool. It maintains a group of threads internally. Each thread (NioEventLoop) is responsible for processing events on multiple channels, and a Channel corresponds to only one thread.

ChannelHandler

ChannelHandler is an interface that processes I/O events or intercepts I/O operations and forwards them to the next handler in its ChannelPipeline(business processing chain).

ChannelHandler itself does not provide many methods, because the interface has a number of methods that need to be implemented, which can be subclassed during use:

  • ChannelInboundHandler Is used to handle inbound I/O events
  • ChannelOutboundHandler Is used to process outbound I/O operations

Or use the following adapter classes:

  • ChannelInboundHandlerAdapter handles inbound I/O events
  • ChannelOutboundHandlerAdapter used to handle the outbound I/O operations
  • ChannelDuplexHandler is used to handle inbound and outbound events

ChannelHandlerContext

Saves all context information associated with a Channel, along with a ChannelHandler object

ChannelPipline

Saves a List of ChannelHandlers that handle or intercept inbound events and outbound operations for a Channel. ChannelPipeline implements an advanced form of intercepting filter pattern that gives the user complete control over how events are handled and how the various Channelhandlers in a Channel interact with each other.

ChannelPipline in Netty’s Javadoc4.1 shows how ChannelHandler handles I/O events in ChannelPipeline. I/O events are handled by ChannelInboundHandler or ChannelOutboundHandler. And by calling ChannelHandlerContext defined event propagation method (for example ChannelHandlerContext. FireChannelRead (Object) and ChannelOutboundInvoker. Write (Object ) to its nearest handler.

I/O Request via Channel or ChannelHandlerContext | +---------------------------------------------------+---------------+  | ChannelPipeline | | | \|/ | | +---------------------+ +-----------+----------+ | | | Inbound Handler N | | Outbound Handler 1 | | | +----------+----------+ +-----------+----------+ | | /|\ | | | | \|/ | | +----------+----------+ +-----------+----------+ | | | Inbound Handler N-1 | | Outbound Handler 2 | | | +----------+----------+ +-----------+----------+ | | /|\ . | | . . | | ChannelHandlerContext.fireIN_EVT() ChannelHandlerContext.OUT_EVT()| | [ method call] [method call] | | . . | | . \|/ | | +----------+----------+ +-----------+----------+ | | | Inbound Handler 2 | | Outbound Handler M-1 | | | +----------+----------+ +-----------+----------+ | | /|\ | | | | \|/ | | +----------+----------+ +-----------+----------+ | | | Inbound Handler 1 | | Outbound Handler M | | | + -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- + + -- -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- + | | | / \ | | + -- -- -- -- -- -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - + -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- + | \ | / +---------------+-----------------------------------+---------------+ | | | | | [ Socket.read() ] [ Socket.write() ] | |  | | Netty Internal I/O Threads (Transport Implementation) | +-------------------------------------------------------------------+ 123456789101112131415161718192021222324252627282930313233343536373839Copy the code

Inbound events are handled by the bottom-up inbound handler, as shown on the left. Inbound Handler handlers typically process inbound data generated by the I/O thread at the bottom of the diagram. Inbound data is usually read remotely from an actual input operation (such as socketchannel.read (ByteBuffer)).

Outbound events are handled up and down, as shown on the right. Outbound Handler handlers typically generate or transform outbound transports, such as write requests. I/O threads typically perform actual output operations, such as socketChannel.write (ByteBuffer).

In Netty, each Channel has only one Channel pipeline corresponding to it, and their composition relationship is as follows:

A Channel contains a ChannelPipeline, which maintains a two-way list of ChannelHandlerContext, And each ChannelHandlerContext is associated with a ChannelHandler. Inbound events and outbound events In a bidirectional list, inbound events are passed from the head to the last inbound handler, and outbound events are passed from tail to the last outbound handler. The two types of handlers do not interfere with each other.

Working Principle framework

The process for initializing and starting the Netty server is as follows:

Public static void main(String[] args) {// Create mainReactor NioEventLoopGroup boosGroup = new NioEventLoopGroup(); NioEventLoopGroup workerGroup = new NioEventLoopGroup(); final ServerBootstrap serverBootstrap = new ServerBootstrap(); NioEventLoopGroup. Group (boosGroup, WorkerGroup) / / set the channel type for NIO type. The channel (NioServerSocketChannel. Class) / / set the connection configuration parameters. The option (ChannelOption SO_BACKLOG, 1024) .childOption(ChannelOption.SO_KEEPALIVE, true) .childOption(ChannelOption.TCP_NODELAY, ChildHandler (new ChannelInitializer<NioSocketChannel>() {@override protected void InitChannel (NioSocketChannel ch) {// Configure the inbound and outbound events channel ch.pipeline().addLast(...) ; ch.pipeline().addLast(...) ; }}); // Bind port int port = 8080; serverBootstrap.bind(port).addListener(future -> { if (future.isSuccess()) { System.out.println(new Date() + ": Port [" + port + "] bound successfully! ); } else {system.err. Println (" port [" + port + "] binding failed!" ); }}); }Copy the code
  • The basic process is as follows:
  • 1 Initially create two NioEventLoopGroups. BoosGroup is used for Accetpt connection establishment events and sends requests, and workerGroup is used for I/O read/write events and service logic
  • 2 Based on ServerBootstrap(server boot class), configure the EventLoopGroup, Channel type, connection parameters, and inbound and outbound event handlers
  • 3 Bind ports and start working

Based on the Netty Reactor model, this paper introduces the server Netty architecture diagram:

The server contains 1 Boss NioEventLoopGroup and 1 Worker NioEventLoopGroup. NioEventLoopGroup is equivalent to an event loop group, which contains multiple event loops NioEventLoop. Each NioEventLoop contains one selector and one event loop thread.

Each Boss NioEventLoop executes a task consisting of three steps:

  • 1 Poll the Accept event
  • 2 Process accept I/O event, establish connection with Client, generate NioSocketChannel, and register NioSocketChannel with Selector of Worker NioEventLoop *3 Process tasks in the task queue. RunAllTasks. The tasks in the task queue include tasks performed by users calling Eventloop. execute or Schedule, or tasks submitted to the Eventloop by other threads.

Each Worker NioEventLoop consists of three steps:

  • 1 Poll read and write events.
  • Two I/O events, namely read and write events, are processed when NioSocketChannel readable and writable events occur
  • 3 runAllTasks to process tasks in the task queue.

There are three typical application scenarios for tasks in the task queue

  • 1 User programs customize common tasks
ctx.channel().eventLoop().execute(new Runnable() { @Override public void run() { //... }}); 1234567Copy the code
  • 2. Various methods of calling channel by non-current reactor thread for example, in the business thread of push system, find the corresponding channel reference according to the user’s id, and then call write class method to push message to the user, this scenario will be entered. The final write is submitted to the task queue and consumed asynchronously.
  • 3 User-defined scheduled task
ctx.channel().eventLoop().schedule(new Runnable() {
    @Override
    public void run() {

    }
}, 60, TimeUnit.SECONDS);

1234567
Copy the code

conclusion

Currently, the mainstream version of ForkJoinPool is recommended for stable use. The use of ForkJoinPool in Netty5 increases code complexity, but does not provide significant performance improvements. This version is not recommended and is not available for download.

The barrier to entry for Netty is relatively high because there is less information on the subject, not because it is difficult, but because you can figure Netty out like Spring. Before learning, it is recommended to understand the whole framework principle structure and operation process, so that many detours can be avoided.

Source: author: thinkwon thinkwon.blog.csdn.net/article/det…