Break the JAVA NIO technology barrier

Source: hiddenpps.blog.csdn.net/article/det…

There are more and more scenarios using NIO, and many technology frameworks on the web use NIO to some extent, such as Tomcat and Jetty. Learning and mastering NIO technology is no longer a plus for a JAVA siege lion, but a must. In addition, nowadays, the Internet interview will involve NIO or AIO questions (AIO will be discussed next time, this paper mainly focuses on NIO). A good command of NIO can also help you get a better offer. The key that drives the blogger to write this article is that there are not many articles about NIO on the Internet, and there are few cases. Aiming at this feature, this paper mainly describes the usage of NIO through practical cases, and each case has been tested. Through her own understanding and some cases, the blogger hopes to give you a reference when learning NIO. The blogger has limited ability, the article has shortcomings welcome place.

An overview of the

NIO has three core parts: channels, buffers, and selectors. Whereas traditional IO operates on byte streams and character streams, NIO operates on channels and buffers, where data is always read from a Channel into a Buffer or written from a Buffer into a Channel. A Selector listens for events on multiple channels (such as a connection opening, data arrival). Thus, a single thread can listen on multiple data channels.

The first big difference between NIO and traditional IO is that WHILE IO is stream-oriented, NIO is buffer-oriented. Java IO stream-oriented means that one or more bytes are read from the stream at a time until all bytes are read without being cached anywhere. In addition, it cannot move data back and forth in a stream. If you need to move data read from the stream back and forth, you need to cache it into a buffer first. NIO’s buffer-oriented approach is slightly different. The data is read into a buffer that it processes later and can be moved back and forth in the buffer as needed. This adds flexibility to the process. However, you also need to check that the buffer contains all the data you need to process. Also, you need to make sure that when more data is read into the buffer, you don’t overwrite the unprocessed data in the buffer.

The various streams of IO are blocked. This means that when a thread calls read() or write(), the thread blocks until some data is read, or data is written entirely. The thread can’t do anything else in the meantime. NIO’s non-blocking mode allows a thread to send a request from a channel to read data, but it only gets what is currently available, and if no data is currently available, it gets nothing. Instead of keeping the thread blocked, it can continue doing other things until the data becomes readable. The same is true for non-blocking writes. A thread requests to write some data to a channel, but without waiting for it to write completely, the thread can do something else in the meantime. Threads typically spend the idle time of non-blocking IO performing IO operations on other channels, so a single thread can now manage multiple input and output channels.

Channel

First, let’s talk about Channel, which is mostly translated as “Channel” in China. A Channel is about the same level as a Stream in IO. Only the Stream is one-way, for example: InputStream, OutputStream. A Channel is bidirectional and can be used for both read and write operations. The main implementations of channels in NIO are:

FileChannel
DatagramChannel
SocketChannel
ServerSocketChannel

It can correspond to file IO, UDP, and TCP (Server and Client) respectively. The following examples are basically about these four types of channels.

Buffer

The key Buffer implementations in NIO are: ByteBuffer, CharBuffer, DoubleBuffer, FloatBuffer, IntBuffer, LongBuffer, ShortBuffer, corresponding to the basic data types respectively: Byte, char, double, float, int, long, short. Of course, there are MappedByteBuffer, HeapByteBuffer, DirectByteBuffer and so on in NIO.

Selector

Selector runs on a single thread to handle multiple channels, which can be handy if your application has multiple channels open, but traffic per connection is low. For example, in a chat server. To use a Selector, you register a Channel with a Selector and then call its select() method. This method blocks until a registered channel is ready for an event. Once the method returns, the thread can process the events, such as new connections coming in, data receiving, and so on.

FileChannel

After reading the above statement, it was very confusing for the students who came into contact with NIO for the first time. They only mentioned some concepts and didn’t remember anything, let alone how to use them. To give you a sense of what NIO is all about, we start by comparing traditional IO with the new NIO.

Traditional IO vs. NIO

First, case 1 uses a FileInputStream to read the contents of a file:

public static void method2(){ InputStream in = null; try{ in = new BufferedInputStream(new FileInputStream("src/nomal_io.txt")); byte [] buf = new byte[1024]; int bytesRead = in.read(buf); while(bytesRead ! = -1) { for(int i=0; i<bytesRead; i++) System.out.print((char)buf[i]); bytesRead = in.read(buf); } }catch (IOException e) { e.printStackTrace(); }finally{ try{ if(in ! = null){ in.close(); } }catch (IOException e){ e.printStackTrace(); }}} 1234567891011121314151617181920212223242526Copy the code

Output result :(omitted)

Case is corresponding to the NIO (operated by RandomAccessFile here, of course, can also pass a FileInputStream. GetChannel () operation) :

    public static void method1(){
        RandomAccessFile aFile = null;
        try{
            aFile = new RandomAccessFile("src/nio.txt","rw");
            FileChannel fileChannel = aFile.getChannel();
            ByteBuffer buf = ByteBuffer.allocate(1024);

            int bytesRead = fileChannel.read(buf);
            System.out.println(bytesRead);

            while(bytesRead != -1)
            {
                buf.flip();
                while(buf.hasRemaining())
                {
                    System.out.print((char)buf.get());
                }

                buf.compact();
                bytesRead = fileChannel.read(buf);
            }
        }catch (IOException e){
            e.printStackTrace();
        }finally{
            try{
                if(aFile != null){
                    aFile.close();
                }
            }catch (IOException e){
                e.printStackTrace();
            }
        }
    }
123456789101112131415161718192021222324252627282930313233
Copy the code

By carefully comparing Case 1 and Case 2, you should be able to get an idea, at the very least, of how NIO is implemented. With a general impression we can move on to the next step.

The use of the Buffer

From case 2, it can be concluded that using Buffer generally follows the following steps:

Allocate space (ByteBuffer buf = ByteBuffer. Allocate (1024); There’s also an allocateDirector which I’ll talk about later.)
Write data to Buffer(int bytesRead = filechannel.read (buf)😉
Call the filp() method (buf.flip();)
Read data from Buffer (system.out.print ((char)buf.get()))
Call either the clear() method or the Compact () method

A Buffer is, as its name suggests, a container, a contiguous array. Channel Provides a Channel for reading data from files and networks. However, the read and write data must pass through Buffer. The diagram below:

Write data to Buffer:

Write from Channel to Buffer (Filechannel.read (buf))
Buffer’s put() method (buf.put(…)) )

Read data from Buffer:

Read from Buffer to Channel (channel.write(buf))
Read data from Buffer using get() (buf.get())

A Buffer can be thought of simply as a list of elements of a basic data type that holds the current position state of the data through several variables: Capacity, position, limit, mark:

The index

instructions

capacity

The total length of the buffer array

position

The location of the next data element to operate on

limit

Position of the next non-operable element in the buffer array: limit<=capacity

mark

Record the previous position of the current position or -1 by default

For example, we create an 11-byte array buffer using the bytebuffer.allocate (11) method. The initial state is shown above, position is 0, and capacity and limit are both array lengths by default. When we write 5 bytes, the change looks like this:

We need to write 5 bytes of buffer data to the Channel’s communication Channel, so we call bytebuffer.flip () and change position back to 0 and limit to the previous position:

At this point, the underlying operating system can correctly read the five-byte data from the buffer and send it. The clear() method is called again before the next write, and the index position of the buffer is back to its original position.

The clear() method is called: Position is set back to 0 and limit is set to Capacity. In other words, the Buffer is cleared. If there is some unread data in the Buffer, call the clear() method and the data will be “forgotten,” meaning there will no longer be any markers telling you which data has been read and which has not. If you still have unread data in the Buffer and need it later, but want to write some data first, use the Compact () method. The compact() method copies all unread data to the start of the Buffer. Position is then set directly after the last unread element. The limit property is set to Capacity as in the clear() method. The Buffer is now ready to write data, but does not overwrite unread data.

A specific position in a Buffer can be marked by calling buffer.mark (), which can later be restored by calling buffer.reset (). The buffer.rewind () method sets position back to 0, so you can re-read all the data in Buffer. Limit remains the same and still indicates how many elements can be read from Buffer.

SocketChannel

With FileChannel and Buffer out of the way, you should be familiar with the use of Buffer, but here we continue our discussion of NIO using SocketChannel. Part of NIO’s power comes from the non-blocking nature of channels; some operations on sockets may block indefinitely. For example, a call to the Accept () method might block waiting for a client connection; A call to the read() method might block because there is no data to read until new data arrives at the other end of the connection. In general, I/O calls such as creating/receiving connections or reading and writing data can block and wait indefinitely until something happens to the underlying network implementation. Slow, lossy networks, or simple network failures can cause arbitrary delays. Unfortunately, you can’t tell if a method is blocking until you call it. An important feature of NIO’s channel abstraction is that its blocking behavior can be configured to achieve a non-blocking channel.

channel.configureBlocking(false)
1
Copy the code

Calling a method on a non-blocking channel always returns immediately. The return value of such a call indicates the degree to which the requested operation has been completed. For example, calling the Accept () method on a non-blocking ServerSocketChannel returns the client SocketChannel if a connection request comes in, or null otherwise.

Here is an example of a TCP application where the client uses NIO and the server still uses IO. Client code (Case 3) :

public static void client(){ ByteBuffer buffer = ByteBuffer.allocate(1024); SocketChannel socketChannel = null; try { socketChannel = SocketChannel.open(); socketChannel.configureBlocking(false); SocketChannel. Connect (new InetSocketAddress (8080) "10.10.195.115,"); if(socketChannel.finishConnect()) { int i=0; while(true) { TimeUnit.SECONDS.sleep(1); String info = "I'm "+i+++"-th information from client"; buffer.clear(); buffer.put(info.getBytes()); buffer.flip(); while(buffer.hasRemaining()){ System.out.println(buffer); socketChannel.write(buffer); } } } } catch (IOException | InterruptedException e) { e.printStackTrace(); } finally{ try{ if(socketChannel! =null){ socketChannel.close(); } }catch(IOException e){ e.printStackTrace(); }}} 12345678910111213141516171819202122232425262728293031323334353637383940Copy the code

Server code (Case 4) :

public static void server(){ ServerSocket serverSocket = null; InputStream in = null; try { serverSocket = new ServerSocket(8080); int recvMsgSize = 0; byte[] recvBuf = new byte[1024]; while(true){ Socket clntSocket = serverSocket.accept(); SocketAddress clientAddress = clntSocket.getRemoteSocketAddress(); System.out.println("Handling client at "+clientAddress); in = clntSocket.getInputStream(); while((recvMsgSize=in.read(recvBuf))! =-1){ byte[] temp = new byte[recvMsgSize]; System.arraycopy(recvBuf, 0, temp, 0, recvMsgSize); System.out.println(new String(temp)); } } } catch (IOException e) { e.printStackTrace(); } finally{ try{ if(serverSocket! =null){ serverSocket.close(); } if(in! =null){ in.close(); } }catch(IOException e){ e.printStackTrace(); }}} 12345678910111213141516171819202122232425262728293031323334353637Copy the code

Output result :(omitted)

According to the case analysis, summarize the usage of SocketChannel. Open a SocketChannel:

socketChannel = SocketChannel.open(); SocketChannel. Connect (new InetSocketAddress (8080) "10.10.195.115,"); 12Copy the code

Close:

socketChannel.close();
1
Copy the code

Read data:

                    String info = "I'm "+i+++"-th information from client";
                    buffer.clear();
                    buffer.put(info.getBytes());
                    buffer.flip();
                    while(buffer.hasRemaining()){
                        System.out.println(buffer);
                        socketChannel.write(buffer);
                    }
12345678
Copy the code

Note that the socketChannel.write () method is called in a while loop. The Write() method does not guarantee how many bytes will be written to SocketChannel. So, we call write() repeatedly until the Buffer has no bytes to write. In non-blocking mode, the read() method may return before any data has been read. So you need to pay attention to its int return value, which will tell you how many bytes were read.

TCP server NIO notation

So far, none of the cases we’ve done involve selectors. Take your time with good things. The Selector class can be used to avoid the wasteful “busy etc” methods of blocking clients. For example, consider an IM server. Like QQ or Wangwang, there may be tens of thousands or even tens of millions of clients connected to the server at the same time, but at any moment is only a very small amount of information.

Need to read and distribute. This requires a way to block and wait until at least one channel is available for I/O operations, indicating which channel. NIO’s selectors do just that. A Selector instance can simultaneously check the I/O status of a set of channels. In technical terms, a selector is a multiplexing switch selector, because a selector can manage I/O operations over multiple channels. However, the traditional way to deal with so many clients is to loop through all the clients one by one to see if they have I/O operations. If the current client has I/O operations, the current client may be thrown to a thread pool for processing. If there are no I/O operations, the next polling. When all clients have polled and then start polling from scratch; This method is very stupid and very wasteful of resources, because most clients do not have I/O operations, we also need to check; A Selector, on the other hand, internally manages multiple I/ OS at the same time, so when a channel has an I/O, it notifies the Selector, and the Selector remembers that the channel has an I/O, and it knows what kind of I/O it is, is it a read? Is to write? Or accept new connections; So if you use Selector, it only returns two results, one is zero, which is no client that needs I/O at the time you call it, and the other is a set of clients that need I/O, and then you don’t need to check at all, because it’s going to give you exactly what you want. Such a notification is much more efficient than active polling!

To use a Selector, you create an instance of a Selector (using the static factory method open()) and register it on the channel you want to monitor (note that this is done using the method of a channel, not the method of a Selector). Finally, the select() method of the selector is called. This method blocks waiting until one or more channels are ready for I/O operations or wait times out. The select() method returns the number of channels available for I/O operations. Now, in a single thread, you can check that multiple channels are ready for I/O by calling the SELECT () method. If no channel is ready after a certain amount of time, the select() method returns 0 and allows the program to continue with other tasks.

Here’s how to rewrite the TCP server code to NIO (case 5) :

public class ServerConnect
{
    private static final int BUF_SIZE=1024;
    private static final int PORT = 8080;
    private static final int TIMEOUT = 3000;

    public static void main(String[] args)
    {
        selector();
    }

    public static void handleAccept(SelectionKey key) throws IOException{
        ServerSocketChannel ssChannel = (ServerSocketChannel)key.channel();
        SocketChannel sc = ssChannel.accept();
        sc.configureBlocking(false);
        sc.register(key.selector(), SelectionKey.OP_READ,ByteBuffer.allocateDirect(BUF_SIZE));
    }

    public static void handleRead(SelectionKey key) throws IOException{
        SocketChannel sc = (SocketChannel)key.channel();
        ByteBuffer buf = (ByteBuffer)key.attachment();
        long bytesRead = sc.read(buf);
        while(bytesRead>0){
            buf.flip();
            while(buf.hasRemaining()){
                System.out.print((char)buf.get());
            }
            System.out.println();
            buf.clear();
            bytesRead = sc.read(buf);
        }
        if(bytesRead == -1){
            sc.close();
        }
    }

    public static void handleWrite(SelectionKey key) throws IOException{
        ByteBuffer buf = (ByteBuffer)key.attachment();
        buf.flip();
        SocketChannel sc = (SocketChannel) key.channel();
        while(buf.hasRemaining()){
            sc.write(buf);
        }
        buf.compact();
    }

    public static void selector() {
        Selector selector = null;
        ServerSocketChannel ssc = null;
        try{
            selector = Selector.open();
            ssc= ServerSocketChannel.open();
            ssc.socket().bind(new InetSocketAddress(PORT));
            ssc.configureBlocking(false);
            ssc.register(selector, SelectionKey.OP_ACCEPT);

            while(true){
                if(selector.select(TIMEOUT) == 0){
                    System.out.println("==");
                    continue;
                }
                Iterator<SelectionKey> iter = selector.selectedKeys().iterator();
                while(iter.hasNext()){
                    SelectionKey key = iter.next();
                    if(key.isAcceptable()){
                        handleAccept(key);
                    }
                    if(key.isReadable()){
                        handleRead(key);
                    }
                    if(key.isWritable() && key.isValid()){
                        handleWrite(key);
                    }
                    if(key.isConnectable()){
                        System.out.println("isConnectable = true");
                    }
                    iter.remove();
                }
            }

        }catch(IOException e){
            e.printStackTrace();
        }finally{
            try{
                if(selector!=null){
                    selector.close();
                }
                if(ssc!=null){
                    ssc.close();
                }
            }catch(IOException e){
                e.printStackTrace();
            }
        }
    }
}
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596
Copy the code

Let’s walk through this code slowly.

ServerSocketChannel

Open the ServerSocketChannel:

ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
1
Copy the code

Close the ServerSocketChannel:

serverSocketChannel.close();
1
Copy the code

Listen for incoming connections:

while(true){
    SocketChannel socketChannel = serverSocketChannel.accept();
}
123
Copy the code

ServerSocketChannel can be set to non-blocking mode. In non-blocking mode, the Accept () method returns immediately, or null if no new connection has been entered. Therefore, check whether the returned SocketChannel is null. Such as:

        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
        serverSocketChannel.socket().bind(new InetSocketAddress(9999));
        serverSocketChannel.configureBlocking(false);
        while (true)
        {
            SocketChannel socketChannel = serverSocketChannel.accept();
            if (socketChannel != null)
            {
                // do something with socketChannel...
            }
        }
1234567891011
Copy the code

Selector

Create Selector: Selector Selector = Selector. Open ();

In order to Channel and the Selector, the Channel must be registered with the Selector, through SelectableChannel. The register () method to implement, use case 5 parts of the code:

            ssc= ServerSocketChannel.open();
            ssc.socket().bind(new InetSocketAddress(PORT));
            ssc.configureBlocking(false);
            ssc.register(selector, SelectionKey.OP_ACCEPT);
1234
Copy the code

When used with Selector, a Channel must be in non-blocking mode. This means that FileChannel cannot be used with a Selector because FileChannel cannot switch to non-blocking mode. Socket channels work.

Notice the second parameter to the register() method. This is an “interest set,” meaning what event is of interest when listening to a Channel with Selector. You can listen for four different types of events:

1. Connect
2. Accept
3. Read
4. Write
1234
Copy the code

The channel raised an event meaning that the event is ready. Therefore, a channel that successfully connects to another server is said to be “connection-ready”. A server socket channel that is ready to receive incoming connections is called “receive ready”. A channel with data to read can be said to be “read ready”. A channel waiting for data to be written can be said to be “write ready”.

The four kinds of events are represented by the four constants of SelectionKey:

1. SelectionKey.OP_CONNECT
2. SelectionKey.OP_ACCEPT
3. SelectionKey.OP_READ
4. SelectionKey.OP_WRITE
1234
Copy the code

SelectionKey

When registering a Channel with a Selector, the register() method returns a SelectionKey object. This object contains some properties of interest to you:

Collection of interest
Ready set
Channel
Selector
Attached object (optional)

Interest collection: As described in the Registering channels with Selector section, an interest collection is a collection of events that you select that you are interested in. Interest collections can be read and written by SelectionKey.

The Ready collection is a collection of operations for which the channel is ready. After a Selection, you first access the Ready set. Selection is explained in the next section. The ready collection can be accessed like this:

int readySet = selectionKey.readyOps();
1
Copy the code

You can detect what events or actions are already in place in a channel in the same way you detect interest collections. However, you can also use the following four methods, all of which return a Boolean type:

selectionKey.isAcceptable();
selectionKey.isConnectable();
selectionKey.isReadable();
selectionKey.isWritable();
1234
Copy the code

Accessing channels and selectors from the SelectionKey is easy. As follows:

Channel  channel  = selectionKey.channel();
Selector selector = selectionKey.selector();
12
Copy the code

An object or more information can be attached to the SelectionKey to make it easy to identify a given channel. For example, you can attach a Buffer that is used with a channel, or an object that contains aggregated data. The usage method is as follows:

selectionKey.attach(theObject);
Object attachedObj = selectionKey.attachment();
12
Copy the code

You can also attach objects when you register a Channel with a Selector using the register() method. Such as:

SelectionKey key = channel.register(selector, SelectionKey.OP_READ, theObject);
1
Copy the code

Select channels by Selector

Once one or more channels have been registered with a Selector, several overloaded select() methods can be called. These methods return channels that are ready for the event you are interested in (such as connect, receive, read, or write). In other words, if you are interested in “read-ready” channels, the select() method returns those channels for which the read event is ready.

Here is the select() method:

int select()
int select(long timeout)
int selectNow()

Select () blocks until at least one channel is ready on the event you registered. Select (long timeout) is the same as select(), except that it blocks at most timeout milliseconds. SelectNow () does not block and returns immediately whenever any channel is ready. If no channels have become selectable since the previous selection, this method returns zero. .

The int returned by the select() method indicates how many channels are ready. That is, how many channels have become ready since the select() method was last called. If you call select(), it returns 1 because one channel is ready, and if you call select() again, it returns 1 again if the other channel is ready. If nothing is done to the first ready channel, there are now two ready channels, but only one is ready between each select() method call.

Once the select() method is called and the return value indicates that one or more channels are ready, the ready channels in the selectedKey Set can be accessed by calling the Selector selectedKeys() method. As follows:

Set selectedKeys = selector.selectedKeys();
1
Copy the code

When registering a Channel with a Selector, the channel.register () method returns a SelectionKey object. This object represents the channel registered with that Selector.

Note the keyiterator.remove () call at the end of each iteration. The Selector does not remove the SelectionKey instance from the selected key set itself. It must be removed by itself when the channel is processed. The next time the channel becomes ready, the Selector puts it into the selected key set again.

The channel returned by the selectionkey.channel () method needs to be converted to the type you want to work with, such as ServerSocketChannel or SocketChannel.

For a complete example of using Selector and ServerSocketChannel, see the Selector () method in Case 5.

Memory-mapped file

JAVA process large documents, commonly used BufferedReader, BufferedInputStream such buffer IO classes, but if the file is big, faster way MappedByteBuffer is adopted.

MappedByteBuffer is a file memory mapping scheme introduced by NIO with high read and write performance. The main thing about NIO is that it supports asynchronous operations. One method registers a SOCKET channel with a Selector, and calls its select method from time to time to return the matched SelectionKey, which contains information about SOCKET events. That’s the SELECT model.

SocketChannel reads and writes are handled by a class called ByteBuffer. The class itself is well designed and much more convenient than manipulating byte[] directly. ByteBuffer has two modes: direct/indirect. The most typical (and only one) indirect pattern is the HeapByteBuffer, which operates on heap memory (byte[]). But memory is limited, so what if I want to send a file of 1 GIGAByte? It’s impossible to actually allocate 1 gigabyte of memory. In this case, you must use the “direct” mode, MappedByteBuffer, file mapping.

Let’s pause and talk about memory management for the operating system. General operating system memory is divided into two parts: physical memory; Virtual memory. Virtual memory typically uses page image files, that is, some special file on the hard disk. The operating system is responsible for the page file content, speaking, reading and writing, this process is called “page break/switch”. MappedByteBuffer is also similar, you can put the entire file (no matter how much they file) as a ByteBuffer. MappedByteBuffer Just a special ByteBuffer, which is a subclass of ByteBuffer. The MappedByteBuffer maps files directly to memory (virtual memory, not physical memory). In general, you can map the entire file, or if the file is large, you can map it in segments, just by specifying that part of the file.

concept

FileChannel provides map methods to map files to memory images: MappedByteBuffer map(int mode,long position,long size); It is possible to map the size area of the file starting with position into a memory image file. Mode specifies how the memory image file can be accessed:

READ_ONLY, (read-only) : Attempts to modify the resulting buffer will result in ReadOnlyBufferException being thrown.(mapmode.read_only)
READ_WRITE (read/write) : Changes to the resulting buffer are eventually propagated to the file; The change may not be visible to other programs that map to the same file. (MapMode.READ_WRITE)
PRIVATE: Changes to the resulting buffer are not propagated to the file and are not visible to other programs that map to the same file; Instead, a dedicated copy of the modified portion of the buffer is created. (MapMode.PRIVATE)

MappedByteBuffer is a subclass of ByteBuffer that extends three methods:

Force () : when the buffer is in READ_WRITE mode, this method forcibly writes changes to the buffer to the file.
Load () : loads the contents of a buffer into memory and returns a reference to the buffer;
IsLoaded () : Returns true if the contents of the buffer are in physical memory, false otherwise;

Case comparison

Method3 () uses MappedByteBuffer to read files “SRC /1.ppt” with a size of about 5M. Method4 () corresponds to ByteBuffer.

public static void method4(){ RandomAccessFile aFile = null; FileChannel fc = null; try{ aFile = new RandomAccessFile("src/1.ppt","rw"); fc = aFile.getChannel(); long timeBegin = System.currentTimeMillis(); ByteBuffer buff = ByteBuffer.allocate((int) aFile.length()); buff.clear(); fc.read(buff); //System.out.println((char)buff.get((int)(aFile.length()/2-1))); //System.out.println((char)buff.get((int)(aFile.length()/2))); //System.out.println((char)buff.get((int)(aFile.length()/2)+1)); long timeEnd = System.currentTimeMillis(); System.out.println("Read time: "+(timeEnd-timeBegin)+"ms"); }catch(IOException e){ e.printStackTrace(); }finally{ try{ if(aFile! =null){ aFile.close(); } if(fc! =null){ fc.close(); } }catch(IOException e){ e.printStackTrace(); } } } public static void method3(){ RandomAccessFile aFile = null; FileChannel fc = null; try{ aFile = new RandomAccessFile("src/1.ppt","rw"); fc = aFile.getChannel(); long timeBegin = System.currentTimeMillis(); MappedByteBuffer mbb = fc.map(FileChannel.MapMode.READ_ONLY, 0, aFile.length()); // System.out.println((char)mbb.get((int)(aFile.length()/2-1))); // System.out.println((char)mbb.get((int)(aFile.length()/2))); //System.out.println((char)mbb.get((int)(aFile.length()/2)+1)); long timeEnd = System.currentTimeMillis(); System.out.println("Read time: "+(timeEnd-timeBegin)+"ms"); }catch(IOException e){ e.printStackTrace(); }finally{ try{ if(aFile! =null){ aFile.close(); } if(fc! =null){ fc.close(); } }catch(IOException e){ e.printStackTrace(); }}} 12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061Copy the code

By running in the entry function main() :

        method3();
        System.out.println("=============");
        method4();
123
Copy the code

Output (running on a common PC) :

Read time: 2ms
=============
Read time: 12ms
123
Copy the code

We can see the difference between the two files in the output. An example may be accidental. Replace a 5M file with a 200M file.

Read time: 1ms
=============
Read time: 407ms
123
Copy the code

You can see the gap widening.

Note: MappedByteBuffer has resource release issues: files opened by MappedByteBuffer are closed only for garbage collection, and this point is undefined. In Javadoc it is described here: A mapped byte buffer and the file mapping that it represents the buffer itself is garbage-collected Refer to resources 5 and 6 for details.

Other Functions

After reading the above statement, we have a certain understanding of NIO in detail, the following mainly through a few cases, to illustrate the rest of NIO functions, the following code amount is more, the functional description is less.

Scatter/Gatter

Reading from a Channel is when the data is read and written to multiple buffers. Therefore, a Channel “scatters” the data read from a Channel into multiple buffers.

To gather data into a Channel means to write data from multiple buffers to the same Channel during a write operation. Therefore, a Channel “gathers” data from multiple buffers and sends it to a Channel.

Scatter/Gather is often used when you need to separate the data to be transmitted. For example, when you transmit a message consisting of a header and a body, you might split the body and the header into different buffers so you can easily process the body and the header.

Case study:

import java.io.File; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.IOException; import java.io.OutputStream; import java.nio.ByteBuffer; import java.nio.channels.Channel; import java.nio.channels.FileChannel; public class ScattingAndGather { public static void main(String args[]){ gather(); } public static void gather() { ByteBuffer header = ByteBuffer.allocate(10); ByteBuffer body = ByteBuffer.allocate(10); byte [] b1 = {'0', '1'}; byte [] b2 = {'2', '3'}; header.put(b1); body.put(b2); ByteBuffer [] buffs = {header, body}; try { FileOutputStream os = new FileOutputStream("src/scattingAndGather.txt"); FileChannel channel = os.getChannel(); channel.write(buffs); } catch (IOException e) { e.printStackTrace(); }}} 123456789101112131415161718192021222324252627282930313233343536373839Copy the code

transferFrom & transferTo

FileChannel’s transferFrom() method transfers data from the source channel to the FileChannel.

    public static void method1(){
        RandomAccessFile fromFile = null;
        RandomAccessFile toFile = null;
        try
        {
            fromFile = new RandomAccessFile("src/fromFile.xml","rw");
            FileChannel fromChannel = fromFile.getChannel();
            toFile = new RandomAccessFile("src/toFile.txt","rw");
            FileChannel toChannel = toFile.getChannel();

            long position = 0;
            long count = fromChannel.size();
            System.out.println(count);
            toChannel.transferFrom(fromChannel, position, count);

        }
        catch (IOException e)
        {
            e.printStackTrace();
        }
        finally{
            try{
                if(fromFile != null){
                    fromFile.close();
                }
                if(toFile != null){
                    toFile.close();
                }
            }
            catch(IOException e){
                e.printStackTrace();
            }
        }
    }
12345678910111213141516171819202122232425262728293031323334
Copy the code

The input argument to the position method indicates that data is written to the target file from position, and count indicates the maximum number of bytes transferred. If the free space of the source channel is less than count bytes, the number of bytes transferred is less than the number of bytes requested. Also note that in the implementation of SoketChannel, SocketChannel will only transmit data that is ready at the moment (possibly less than count bytes). Therefore, SocketChannel may not transfer all of the requested data (count bytes) into the FileChannel.

The transferTo() method transfers data from the FileChannel to another channel.

    public static void method2()
    {
        RandomAccessFile fromFile = null;
        RandomAccessFile toFile = null;
        try
        {
            fromFile = new RandomAccessFile("src/fromFile.txt","rw");
            FileChannel fromChannel = fromFile.getChannel();
            toFile = new RandomAccessFile("src/toFile.txt","rw");
            FileChannel toChannel = toFile.getChannel();


            long position = 0;
            long count = fromChannel.size();
            System.out.println(count);
            fromChannel.transferTo(position, count,toChannel);

        }
        catch (IOException e)
        {
            e.printStackTrace();
        }
        finally{
            try{
                if(fromFile != null){
                    fromFile.close();
                }
                if(toFile != null){
                    toFile.close();
                }
            }
            catch(IOException e){
                e.printStackTrace();
            }
        }
    }
123456789101112131415161718192021222324252627282930313233343536
Copy the code

The problem with SocketChannel mentioned above also applies to the transferTo() method. SocketChannel transmits data until the target buffer is full.

Pipe

A Java NIO pipe is a one-way data connection between two threads. Pipe has a source channel and a sink channel. Data will be written to the sink channel and read from the source channel.

public static void method1(){ Pipe pipe = null; ExecutorService exec = Executors.newFixedThreadPool(2); try{ pipe = Pipe.open(); final Pipe pipeTemp = pipe; exec.submit(new Callable<Object>(){ @Override public Object call() throws Exception { Pipe.SinkChannel sinkChannel = pipeTemp.sink(); While (true){timeunit.seconds.sleep (1); String newData = "Pipe Test At Time "+System.currentTimeMillis(); ByteBuffer buf = ByteBuffer.allocate(1024); buf.clear(); buf.put(newData.getBytes()); buf.flip(); while(buf.hasRemaining()){ System.out.println(buf); sinkChannel.write(buf); }}}}); exec.submit(new Callable<Object>(){ @Override public Object call() throws Exception { Pipe.SourceChannel sourceChannel =  pipeTemp.source(); While (true){timeunit.seconds.sleep (1); ByteBuffer buf = ByteBuffer.allocate(1024); buf.clear(); int bytesRead = sourceChannel.read(buf); System.out.println("bytesRead="+bytesRead); while(bytesRead >0 ){ buf.flip(); byte b[] = new byte[bytesRead]; int i=0; while(buf.hasRemaining()){ b[i]=buf.get(); System.out.printf("%X",b[i]); i++; } String s = new String(b); System.out.println("=================||"+s); bytesRead = sourceChannel.read(buf); }}}}); }catch(IOException e){ e.printStackTrace(); }finally{ exec.shutdown(); }} 12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061Copy the code

DatagramChannel

DatagramChannel in Java NIO is a channel for sending and receiving UDP packets. Because UDP is a connectionless network protocol, it cannot be read or written like other channels. It sends and receives packets of data.

public static void reveive(){ DatagramChannel channel = null; try{ channel = DatagramChannel.open(); channel.socket().bind(new InetSocketAddress(8888)); ByteBuffer buf = ByteBuffer.allocate(1024); buf.clear(); channel.receive(buf); buf.flip(); while(buf.hasRemaining()){ System.out.print((char)buf.get()); } System.out.println(); }catch(IOException e){ e.printStackTrace(); }finally{ try{ if(channel! =null){ channel.close(); } }catch(IOException e){ e.printStackTrace(); } } } public static void send(){ DatagramChannel channel = null; try{ channel = DatagramChannel.open(); String info = "I'm the Sender!" ; ByteBuffer buf = ByteBuffer.allocate(1024); buf.clear(); buf.put(info.getBytes()); buf.flip(); Int bytesSent = channel.send(buf, new InetSocketAddress("10.10.195.115",8888)); System.out.println(bytesSent); }catch(IOException e){ e.printStackTrace(); }finally{ try{ if(channel! =null){ channel.close(); } }catch(IOException e){ e.printStackTrace(); }}} 12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152Copy the code

Break the JAVA NIO technology barrier

An overview of the

Channel

Buffer

Selector

FileChannel

Traditional IO vs. NIO

The use of the Buffer

SocketChannel

TCP server NIO notation

ServerSocketChannel

Selector

SelectionKey

Select channels by Selector

Memory-mapped file

concept

Case comparison

Other Functions

Scatter/Gatter

transferFrom & transferTo

Pipe

DatagramChannel

Related Posts

TinyIM, an open source IM that supports chatbots

Is the service discovery mechanism SPI a destroyer? !

Scala + : type inference, list manipulation and for loop