while true  
{  
    for i in stream[]  
    {  
        if i has data  
        read until unavailable  
    }  
}  
Copy the code

We can process multiple streams by simply querying all streams from beginning to end, which is a bad idea because if all streams have no I/O events, we waste CPU time slices. As one scientist said, all computer problems can be solved by adding an intermediate layer. Again, to avoid CPU idling here, instead of having the thread check for events in the stream itself, we introduced a proxy (first select, then poll), which was great. It can observe I/O events for many streams at the same time. If there are no events, the proxy will block and threads will not poll one by one, with the following pseudocode:

While true {select(streams[]) // This step dies here, knowing that a stream has I/O events, For I in streams[] {if I have data read until unavailable}}Copy the code

The problem is that we only know from select that an I/O event occurred, but we don’t know which streams it was (there could be one, more, or even all of them). We have to poll all streams indiscriminately to find the ones that can read or write data, and then operate on them. So select has O(n) undifferentiated polling complexity, and the more streams that are processed at the same time, the longer the undifferentiated polling time.

Epoll can be understood as an event poll. Unlike busy polling and undifferentiated polling, epoll notifies us of which FLOW has what I/O event. So we say that epoll is actually event-driven (each event is associated with a FD) and that our operations on these streams make sense. (complexity reduced to O(1))

while true  
{  
    active_stream[] = epoll_wait(epollfd)  
    for i in active_stream[]  
    {  
        read or write till  
    }  
}  
Copy the code

The main difference between SELECT and epoll is that select only tells you that a certain number of streams have events, and you have to poll them one by one. Epoll tells you the events that happened, and then it automatically locates which stream. It cannot be said that epoll is a qualitative leap compared with Select. I think it is also an idea of sacrificing space for time. After all, hardware is getting cheaper and cheaper now.

Select,poll,epoll,poll,epoll

The most widespread model is the blocking I/O model, where all sockets are blocked by default. The process invokes the recvFROM system call, which blocks and does not return until data is copied to the process buffer (or, of course, when the system call is interrupted).

This model uses the SELECT and poll functions, which also block the process. Select blocks first and returns only when there is an active socket. However, unlike blocking I/O, these two functions can block multiple I/ OS at the same time, and can detect multiple READ and write I/ OS simultaneously. Until data is readable or writable (that is, listening on multiple sockets). After the select is called, the process blocks, the kernel monitors all the select sockets, and when data is ready for any socket, the SELECT returns the socket readable, and we can call recvFROM to process the data. Because blocking I/O blocks only one I/O operation, the I/O multiplexing model blocks multiple I/O operations, it is called multiplexing.

Signal Driven I/O (SIGIO)

First we allow the socket to do signal-driven I/O and install a signal handler. The process continues to run without blocking. When the data is ready, the process receives a SIGIO signal and can process the data by calling the I/O operation function in the signal handler function. When the datagram is ready to be read, the kernel generates a SIGIO signal for the process. We can then either call recvFROM in the signal handler to read the datagram and notify the main loop that the data is ready for processing, or we can immediately notify the main loop to read the datagram. Regardless of how SIGIO signals are processed, the advantage of this model is that the process can continue executing without being blocked while waiting for the datagram to arrive (phase 1). Select blocking and polling are eliminated and active sockets are handled by registered handlers.

\

5, Asynchronous I/O model (AIO, asynchronous I/O)

Once the process initiates the read operation, it can immediately start doing other things. On the other hand, from the kernel’s point of view, when it receives an asynchronous read, it first returns immediately, so no blocks are generated for the user process. The kernel then waits for the data to be ready and copies the data to the user’s memory. When this is done, the kernel sends a signal to the user process telling it that the read operation is complete.

This model works by telling the kernel to start an operation and notifying us when the entire operation (including the second phase, which is copying data from the kernel into the process buffer) is complete.

The difference between this model and the previous one is that signal-driven I/O lets the kernel tell us when an I/O operation can be started, whereas asynchronous I/O lets the kernel tell us when the I/O operation is complete.

Analysis of high performance IO model

Server-side programming often needs to construct high-performance IO models. There are four common IO models:

(1) Blocking IO: the traditional IO model.

(2) Synchronizing non-blocking IO: All created sockets are blocking by default. Non-blocking IO requires that the socket be set to NONBLOCK. Note that NIO is not the NIO (New IO) library for Java.

(3) IO Multiplexing: namely the classic Reactor design pattern, Selector in Java and Epoll in Linux are both this model.

Asynchronous IO is a classic Proactor design pattern, also known as Asynchronous non-blocking IO.

For ease of description, we use the read operation of IO as an example.

Block I/O synchronously

The synchronous blocking IO model is the simplest IO model in which user threads are blocked while the kernel is performing IO operations.

Figure 1 Synchronous blocking IO

As shown in Figure 1, the user thread initiates an IO read through the system call read, moving from user space to kernel space. The kernel waits until the packet arrives, then copies the received data into user space to complete the read operation.

Pseudocode for user threads using the synchronous blocking IO model is described as follows:

{

read(socket, buffer);

process(buffer);

}
Copy the code

The user waits for read to read data from the socket into the buffer before processing the received data. During the I/O request process, the user thread is blocked. As a result, the user cannot do anything during the I/O request, resulting in insufficient CPU utilization.

2. Synchronize non-blocking IO

Synchronizing non-blocking I/OS sets the socket to NONBLOCK on the basis of synchronizing blocking I/OS. This allows the user thread to return immediately after making an IO request.

Figure 2 Synchronizing non-blocking IO

As shown in Figure 2, since the socket is non-blocking, the user thread immediately returns an I/O request. However, no data is read. The user thread continuously initiates I/O requests until the data arrives and then reads the data to continue the execution.

Pseudocode for user threads using the synchronous non-blocking IO model is described as follows:

{ while(read(socket, buffer) ! = SUCCESS) ; process(buffer); }Copy the code

That is, the user repeatedly calls read, attempting to read data from the socket, and does not continue processing the received data until the read succeeds. In the whole I/O request process, although the user thread can immediately return after each I/O request, it still needs to continuously poll and repeat the request to wait for data, consuming a large amount of CPU resources. This model is rarely used directly, but the non-blocking IO feature is used in other IO models.

IO multiplexing

IO multiplexing model is based on the kernel-provided multiplexing function SELECT, which can avoid the problem of polling wait in synchronous non-blocking IO model.

Figure 3. Multiway separation function SELECT

As shown in Figure 3, the user first adds the socket for IO operations to the SELECT, then blocks and waits for the SELECT system call to return. When the data arrives, the socket is activated and the select function returns. The user thread formally initiates a read request, reads the data, and continues execution.

In terms of flow, the USE of SELECT for IO requests is not much different from the synchronous blocking model, and even more inefficient with the additional operations of adding monitoring sockets and calling select. However, the biggest advantage of using SELECT is that the user can process I/O requests from multiple sockets simultaneously in a single thread. The user can register multiple sockets, and then continuously call select to read the activated socket, to achieve the purpose of processing multiple I/O requests in the same thread. In the synchronous blocking model, multithreading is necessary to achieve this goal.

The pseudocode for the user thread using the select function is described as follows:

{ select(socket); while(1) { sockets = select(); for(socket in sockets) { if(can_read(socket)) { read(socket, buffer); process(buffer); }}}}Copy the code

The while loop adds the socket to the SELECT monitor, and then calls select all the way through the while to get the activated socket. Once the socket is readable, the read function is called to read the data from the socket.

However, the advantages of using the SELECT function do not end there. Although the above approach allows multiple I/O requests to be processed in a single thread, the process of each I/O request is still blocked (on the SELECT function), and the average time is even longer than the synchronous blocking IO model. CPU utilization can be improved if the user thread registers only the socket or IO requests it is interested in, then goes about its business and waits until the data arrives to process them.

The IO multiplexing model uses the Reactor design pattern to implement this mechanism.

Figure 4. Reactor Design pattern

As shown in figure 4, the EventHandler abstract class represents an IO EventHandler, which has the IO file Handle Handle (obtained via get_handle) and the operations on Handle handle_event (read/write, etc.). Subclasses that inherit from EventHandler can customize the behavior of event handlers. The Reactor class is used to manage EventHandler (register, delete, etc.) and implement an event loop using HANDLE_Events, which continuously calls the select function of the synchronous event multiplexer (usually the kernel) whenever a file handle is activated (read/write, etc.). Select returns (blocks), and HANDLE_EVENTS calls handLE_EVENT of the event handler associated with the file handle.

Figure 5 IO multiplexing

As shown in Figure 5, Reactor can uniformly transfer the polling of user threads for IO operation status to the HANDLE_Events event loop for processing. The user thread registers the event handler and then proceeds to do other work (asynchronously), while the Reactor thread calls the kernel’s SELECT function to check the socket status. When a socket is activated, the corresponding user thread is notified (or the callback function of the user thread is executed) and handLE_EVENT is executed to read and process data. Because the SELECT function blocks, the multiplex IO multiplexing model is also known as the asynchronous blocking IO model. Note that blocking refers to the thread being blocked when the select function is executed, not the socket. In the IO multiplexing model, sockets are set to NONBLOCK, but this does not matter because when the user initiates an I/O request, the data has already arrived and the user thread will not be blocked.

The pseudo-code for user threads using the IO multiplexing model is described as follows:

void UserEventHandler::handle_event() {

if(can_read(socket)) {

read(socket, buffer);

process(buffer);

}

}

 

{

Reactor.register(new UserEventHandler(socket));

}
Copy the code

The user needs to rewrite the HANDLE_EVENT function of EventHandler to read and process data. The user thread only needs to register its EventHandler with Reactor. The pseudo-code for the HANDLE_Events event loop in Reactor is roughly as follows.

Reactor::handle_events() { while(1) { sockets = select(); for(socket in sockets) { get_event_handler(socket).handle_event(); }}}Copy the code

The event loop continuously calls SELECT to retrieve the activated socket, and then executes handle_event according to the EventHandler corresponding to the socket.

IO multiplexing is the most commonly used IO model, but it is not asynchronous enough because it uses a select system call that blocks threads. Therefore, IO multiplexing can only be called asynchronous blocking IO, not true asynchronous IO.

4. Asynchronous I/O

“True” asynchronous IO requires stronger support from the operating system. In the IO multiplexing model, the event loop notifies the user thread of the status event of the file handle, and the user thread reads and processes the data by itself. In the asynchronous IO model, when the user thread receives the notification, the data has been read by the kernel and placed in the buffer specified by the user thread. The kernel notifies the user thread to use the data directly after I/O completion.

The asynchronous IO model implements this mechanism using the Proactor design pattern.

Figure 6. Proactor design pattern

As shown in FIG. 6, Proactor mode and Reactor mode are similar in structure, but differ greatly in the use mode of Client. In the Reactor pattern, the user thread listens by registering an event of interest with the Reactor object, and then calls an event handler when the event is triggered. But Proactor pattern, user threads will AsynchronousOperation (read/write, etc.), Proactor and CompletionHandler registered to AsynchronousOperationProcessor operation is complete. AsynchronousOperationProcessor using the Facade pattern provides a set of asynchronous operations API (read/write, etc.) for the use of the user, when a user thread calls asynchronous API, then continue to perform their tasks. AsynchronousOperationProcessor opens independent kernel threads execute asynchronous operations, true asynchronous. When the asynchronous I/o operation is complete, AsynchronousOperationProcessor will registered user threads with AsynchronousOperation Proactor and CompletionHandler, The CompletionHandler is then forwarded along with the result data of the IO operation to Proactor, which is responsible for calling back the event CompletionHandler handle_event for each asynchronous operation. While each asynchronous operation in the Proactor pattern can be bound to a Proactor object, proActors are generally implemented in the Singleton pattern in operating systems to facilitate centralized distribution of operation completion events.

Figure 7 Asynchronous IO

As shown in Figure 7, in the asynchronous IO model, the user thread initiates a READ request directly using the asynchronous IO API provided by the kernel and immediately returns to continue executing the user thread code. At this point, however, the user thread has registered the invocation AsynchronousOperation and CompletionHandler into the kernel, and the operating system starts a separate kernel thread to handle the I/O operations. When the read request arrives, the kernel reads the data from the socket and writes it to a user-specified buffer. Finally, the kernel distributes the read data and the user thread’s registered CompletionHandler to the internal Proactor, which notifts the user thread of the I/O completion (usually by calling the user thread’s registered completion event handler) to complete the asynchronous I/O.

The pseudocode for user threads using the asynchronous IO model is described as follows:

void UserCompletionHandler::handle_event(buffer) {

process(buffer);

}

 

{

aio_read(socket, new UserCompletionHandler);

}
Copy the code

The user needs to rewrite the handLE_EVENT function of CompletionHandler to process the data. The parameter buffer represents the data prepared by Proactor. The user thread directly calls the asynchronous IO API provided by the kernel. And register the overwritten CompletionHandler.

Compared with the IO multiplexing model, asynchronous IO is not very common, many high-performance concurrent services using IO multiplexing model + multithreaded task processing architecture can basically meet the needs. In addition, the current operating system for asynchronous IO support is not particularly perfect, more is to use IO multiplexing model to simulate asynchronous IO (IO event trigger does not directly notify the user thread, but after reading and writing data into the user specified buffer). Asynchronous IO has been supported since Java7, and interested readers can try it out.

Reference: Analysis of high-performance IO model

Reference: what is IO multiplexing, understand IO multiplexing

Reference: Understanding synchronization, asynchrony and blocking, non-blocking

\