Let’s start by discussing the differences between BIO, NIO, and AIO in terms of programming models.

The BIO (Blocking I/O) API is designed to block program calls. Such as:

byte a = readKey()
Copy the code

Suppose the readKey method reads a key from the keyboard. If the design is non-blocking I/O, the readKey will not block the current thread. You might ask: What if the user doesn’t have a button? In a blocking I/O design, the thread blocks and waits for the user to press a key if the user does not press a key. In a non-blocking I/O design, the thread does not block and no key returns a null value, such as NULL.

Finally, AIO (Asynchronous I/O), API design creates an additional timeline. Such as

Func callBackFunction(byte keyCode) {// Handle keys} readKey(callBackFunction)Copy the code

In asynchronous I/O, the readKey method returns directly, but with no result. The result needs to be received by a callBackFunction. From this perspective, there are actually two timelines. The first is the main timeline of the program, where readKey is executed up to readKey. The execution of the callBackFunction will be triggered when the user presses the button, that is, the time is not determined. Therefore, the program in the callBackFunction is another timeline, which is also generated based on this reason. We call it asynchrony, which describes the phenomenon that the timeline cannot be synchronized. You don’t know when the callbackFunction is going to execute.

However, we often say that the x language provides asynchronous I/O, not just the way the above program is written, which creates a problem called callback hell, essentially the asynchronous program’s timeline is distorted, resulting in high maintenance costs.

request("/order/123", (data1) -> {

  / /..

  request("/product/456", (data2) -> {

    // ..

    request("/sku/789", (data3) -> {

      / /...})})})Copy the code

For example, the program above (called callback hell) is expensive to maintain, so when we provide an asynchronous API programming model, we provide a syntax to convert asynchrony into a synchronous program. For example, the following pseudocode:

Future future1 = request("/order/123")

Future future2 = request("/product/456")

Future future3 = request("/sku/789")

// ...

// ...

order = future1.get()

product = future2.get()

sku = future3.get()
Copy the code

The request function is a network call that requests order data with order ID=123. The request function itself does not block and executes immediately, whereas the network call is an asynchronous request that does not end on the next line of Request (“/order/123”) but at some point in the future. Therefore, we encapsulate this asynchronous operation with a Future object. Future.get () is a blocking operation that blocks until the network call returns.

Between request and future.get, there are many other things we can do, such as sending more requests. An operation like a Future that synchronizes asynchronous operations back to the main timeline is called asynchronous transsynchronization, also known as asynchronous programming.

When dealing with network I/O problems, there is another key issue to pay attention to, is the use of data structures.

When working with data structures, think about what model to use for I/O multiplexing.

Suppose you are dealing with a highly concurrent site, with a large number of requests coming to your server every second. How many threads do you use to handle I/O? For scenarios that do not require compression and decompression, the main overhead of processing I/O is copying data. So how many copies can a CPU core make per second?

Copying, in fact, is to copy the data in memory from one address to another address. Combined with DMA, memory mapping and other technologies, copying is very fast. Regardless of DMA and memory mapping, a 3GHz CPU can copy hundreds of megabytes of data per second. Of course, speed is also limited by the speed of memory itself. So overall, I/O does not require a lot of computing resources. Usually when we are dealing with high concurrency, we do not need a large number of threads to do I/O processing.

For most applications, the cost of processing I/O is less than the cost of processing business. Processing highly concurrent services may require a large amount of computing resources. Each transaction may also require more I/O, such as remote RPC calls.

Therefore, when dealing with high concurrency, a common I/O multiplexing pattern is that a small number of threads handle a large number of network receive and send tasks. More threads, usually a thread pool, then handle the specific business work.

In such a mode, a core problem needs to be solved: when the operating system kernel detects that an I/O operation has occurred, how does it tell which thread to call which program?

At this point, an efficient model would require us to register threads, the types of events that threads listen to, and the programs that respond to them with the kernel. Specifically, when a client sends a message to the server, we need to know as soon as possible which thread cares about the message (processing the data). Epoll, for example, is such a model, with a red-black tree inside. Specifically, we can see that the file descriptor forms a red-black tree, and the nodes of the red-black tree hold the thread corresponding to the file descriptor, the thread-listening event type, and the corresponding program.

What does BIO, AIO and NIO have to do with all this? There are two connections here.

The first is that no matter which programming model needs to use buffers, that is, BIO, AIO, NIO all need buffers, so it matters a lot. Whenever we use any programming model, if we don’t use buffers internally, we must add buffers externally. Another connection is that registration + notification push methods like ePoll can save us a lot of time in locating specific threads and event types. This is a general technique, not one specific I/O model.

However, from the perspective of capability analysis, using a model like epoll, there is no need to block the I/O processing thread, because the operating system will continuously push the events that need to be responded to to the processing thread, so you can consider not blocking the processing thread (such as NIO).

A thread is the smallest unit of execution. In I/O multiplexing, a single thread processes a large number of I/ OS. There is also a model for executing programs called coroutines, which are lightweight threads. The operating system allocates execution resources to threads and then schedules them to run. If you want to implement coroutines, you create smaller units of execution on top of the execution resources allocated to the thread. Coroutines are not scheduled by the operating system and share the execution resources of the thread.

The point of I/O multiplexing is to reduce the cost of switching between threads. So by design, threads and coroutines are the same as long as a single thread handles a lot of I/O work. If a single thread processes a large amount of I/O, the use of coroutines also depends on the ability of the corresponding thread to execute the coroutine.