First of all, this article is prepared for Netty novices, so every detail of the steps will be listed, old hands do not have to waste time in my article, or you will think I ink.

One, foreword

Netty is an open source asynchronous event-driven network application framework for rapid development of maintainable high-performance protocol servers and clients.

Netty was founded by Trustin Lee, a Korean who now works for Line in Korea. Mina, one of Netty’s most popular apps, was also created by lee.

Netty’s current project leader is German Norman maurer (formerly at Redhat, full time Netty developer), the author of Netty in action, currently a senior engineer at apple, and a frequent participant in Netty related technical conferences. The two bulls look like this:

Netty’s advantages are simple: simple to use, powerful, powerful performance.

Netty features:

  • High concurrency: Netty is a network communication framework based on Nonblocking IO (NIO), compared to BIO (Blocking I/O), its concurrency performance is greatly improved.

  • Fast transmission: Netty’s transmission relies on the zero-copy feature to minimize unnecessary memory copies and achieve more efficient transmission.

  • Encapsulate: Netty encapsulates many of the details of NIO operations and provides an easy-to-use invocation interface.

Netty’s advantages:

  • Simple to use: Encapsulates many details of NIO, making it easier to use.

  • Powerful: preset a variety of encoding and decoding functions, support a variety of mainstream protocols.

  • Strong customization capability: The communication framework can be flexibly extended through ChannelHandler.

  • High performance: Compared with other mainstream NIO frameworks in the industry, Netty has the best overall performance.

  • Stability: Netty fixed all NIO bugs it had found, allowing developers to focus on the business itself.

  • Active community: Netty is an active open source project with short iteration cycles and fast bug fixes. What is Netty’s high performance?

  • IO thread model: synchronously non-blocking, doing more with the fewest resources.

  • Zero memory copy: Minimizes unnecessary memory copy to achieve more efficient transmission.

  • Memory pool design: The allocated memory can be reused, mainly refers to the direct memory. The internal implementation manages memory allocation with a binary lookup tree.

  • Serialization handles reads and writes: Avoids the performance overhead of using locks.

  • High-performance serialization protocol: Supports high-performance serialization protocols such as Protobuf.

What is the difference between BIO, NIO and AIO?

These three concepts correspond to three communication models: blocking, non-blocking, and non-blocking asynchronous. The concept is not written here. Many blogs on the Internet say that Netty corresponds to NIO, which can be either NIO or AIO.

  • BIO: One thread is connected to the server. When the client has a connection request, the server needs to start a thread to process it. Pseudo asynchronous IO: Request connections are put into a thread pool, one-to-many, but threads are still a valuable resource.

  • NIO: One thread per request, but all connection requests sent by the client are registered with the multiplexer. The multiplexer polls the connection for I/O requests and starts a thread for processing.

  • AIO: a valid request is made to a thread. The CLIENT’s I/O request is completed by the OS and then the server application is notified to start the thread for processing.

BIO is flow-oriented, NIO is buffer-oriented; The BIO streams are blocked. NIO is non-blocking; BIO’s Stream is one-way, while NIO’s channel is bidirectional.

NIO features: event-driven model, single-thread processing multi-task, non-blocking I/O, I/O read and write no longer blocks, but returns 0, block-based transmission than stream-based transmission more efficient, more advanced IO functions zero-copy, IO multiplexing greatly improves the scalability and practicality of Java network applications. Based on the Reactor thread model.

What can Netty do?

Learning skills is all about being able to apply them to the actual work. No one is learning for the sake of learning and playing. So what can Netty do? Mainly in two aspects:

Now the Internet of Things is everywhere, a large number of projects are involved in the application of sensors and server-side data communication, Netty as a basic communication component, can easily solve the previously high threshold of communication system development, you no longer have to figure out how to parse all kinds of simple or complex communication protocols. Programmers who have experience in this area will have a deeper, or more memorable, experience.

Now Internet system standards are high concurrent, distributed, micro service, all kinds of news, Netty applied in this type of architecture is like a duck to water, if you are on the current application server, then can completely based on Netty to implement your own HTTP server, FTP server, UDP server, the RPC server, WebSocket server, Redis Proxy server, MySQL Proxy server and so on.

What are the benefits of mastering Netty?

The immediate benefit: the opportunity to get into a big factory and earn a high salary, many of the industry’s leading companies are looking for senior/senior Java engineers who require proficiency in, or familiarity with Netty.

The list could go on, on…

As a Java student, if you haven’t studied Netty, then your use and understanding of the Java language is only superficial. You can do SSH, write a few MVC’s, access databases and caches, and that’s just what a beginner or intermediate Java programmer does. Netty is definitely a must pass if you want to get to the next level of Java server knowledge.

The indirect benefit is that many open source frameworks use Netty, and if you know Netty, you have the foundation to analyze them and become a technology champion.

What are some of these open source frameworks? A brief list of some typical, as follows:

  • Ali distributed service framework Dubbo RPC framework;

  • RocketMQ, Taobao’s messaging middleware;

  • Avro’s RPC framework, a high-performance communication and serialization component of Hadoop;

  • Open source cluster computing framework Spark;

  • Distributed computing framework Storm;

  • Concurrent and distributed applications Akka;

  • The list is still very, very long…

Four, start dry, to achieve a simple example of the transfer string

Before you get started, it’s important to know the basic concepts necessary, otherwise the code will be knocked down, the functionality will be implemented, but you will still have no idea about Netty, which is not the purpose of this article.

This example needs to use the basic knowledge mainly has the following aspects of the east, these knowledge points had better have a general understanding, otherwise, see the example will have certain difficulties.

  • Master Java basics
  • Master the basics of Maven
  • Familiar with IntelliJ IDEA integrated development tool, short for IDEA
  • Know the basic concepts of TCP and Socket

1. Netty components

  • I/O: Various streams (files, arrays, buffers, pipes…) Processing (input/output).

  • Channel: a Channel representing a connection. Each Client pair corresponds to a specific Channel.

  • ChannelPipeline: Chain of responsibility, each Channel has one and only one corresponding Channel, which contains various handlers.

  • Handler: Used to process inbound and outbound messages and corresponding events to achieve our own business logic.

  • EventLoopGroup: AN I/O thread pool that processes I/O events corresponding to channels.

  • ServerBootstrap: Start secondary objects on the server side.

  • Bootstrap: The client starts the auxiliary object.

  • ChannelInitializer: Channel initializer.

  • ChannelFuture: represents the execution result of an I/O operation. Through the event mechanism, we can get the execution result. By adding listeners, we can perform the desired operation.

  • ByteBuf: Sequence of bytes that operate on the underlying byte array and buffer through ByteBuf.

2. Prepare the basic environment

There are three aspects to prepare the basic environment: JDK installation and environment variable Settings, Maven installation and environment variable Settings, IDEA installation and basic Settings.

2.1 JDK installation and environment variable setting

JDK download, can from the official now, also can download links, on baidu to search the latest version is JDK14, I am here to download JDK8, with which version 8 and 14 doesn’t matter, can be, but to note is that from the Oracle to download the JDK’s website, now need to account, can’t no account of, don’t know doing what east east.

Download from the official website: www.oracle.com, screenshot below:

After downloading, install Next, create Java environment variables Settings, right-click [computer]–>[Properties]–>[Advanced System Settings]–>[Environment Variables]–>[System variables], screenshot below:

After the Java environment variables are created, run the java-version command in the DOS window to check whether the environment variables are normal

2.2 Maven installation and environment variable setting

Maven is powerful, but don’t worry. In this example, we just take advantage of the convenience of JAR dependencies and JAR dependency passing, with virtually no learning cost.

Jar dependencies, jar dependencies, jar dependencies, jar dependencies, jar dependencies, jar dependencies

Maven is downloadable, unzipped, and configured with environment variables.

Download address https://downloads.apache.org/maven/maven-3/3.6.3/binaries/

Installation: Download the compressed package, decompress it, and copy the folder to the desired location (such as the root directory of drive C).

MAVEN_HOME = MAVEN_HOME = MAVEN_HOME = MAVEN_HOME = MAVEN_HOME = MAVEN_HOME = MAVEN_HOME

Since it is slow to automatically download jar packages directly from Maven’s central repository, it is common to add ali Cloud’s public repository configuration in Maven’s configuration file, which will significantly speed up the download of JAR packages as follows:

After setting the preceding environment variables, run the mvn-version command in the DOS window to check whether the environment variables are set successfully.

2.3 IDEA installation and basic Settings

The community version does not support HTML, JS, CSS, etc., but for this example, the community version is enough, but if you do not care about the money, you can consider the flagship version, one step in place. In case we have to do WEB system development later, this will save us a lot of trouble.

If you want to install the JDK, you can specify the location of Maven and JDK. If you want to install the JDK, you can specify the location of Maven. [File]–>[setting]–>[Build,Excution,Deployment]–>[Build Tools]–>[Maven]

JDK: [File]–>[Project Structure]–>[Project Setting]–>[Project]

3. Create Maven project in IDEA

New construction

Fill in the package name and project name

Maven configuration

Build project to automatically create Maven dependency files

Configure Netty dependencies in pom.xml

After the above steps, our Maven project has been created, now we can write Netty’s first program, this program is very simple, transfer a string, although the program is very simple, but has been able to reflect the overall flow of Netty communication program development.

Interview questions: a summary of 100 interviews

4. Basic process of Netty development

The basic flow of Netty development is very simple, server and client are the same routine, as follows:

The actual process of Netty development, this is a simplified process, but the general process has been expressed, green represents the client process, blue represents the server process, note the red parts, as shown below:

4.1 Creating a Client Class

Create a Handler

First, we create a Handler class that receives data sent from the server. This is a simplified class that only overwrites the message reading method channelRead0 and the exceptionCaught method.

The client Handler is SimpleChannelInboundHandler inheritance, the class has a lot of methods, heartbeat, timeout detection, connection status, etc.

The code is as follows:

package com.jcj.helloworld; import io.netty.buffer.ByteBuf; import io.netty.channel.ChannelHandler; import io.netty.channel.ChannelHandlerContext; import io.netty.channel.SimpleChannelInboundHandler; import io.netty.util.CharsetUtil; /** * @auther: @date: 2020/6/1 11:12 * @description: General handler, Deal with an I/O * / @ ChannelHandler Sharable public class HandlerClientHello extends SimpleChannelInboundHandler < ByteBuf > { @Override protected void channelRead0(ChannelHandlerContext channelHandlerContext, ByteBuf ByteBuf) throws Exception {/** * @author Jiang Chengjun * @date 2020/6/1 11:17 * @description Processes the received message **/ System.out.println("Message received:"+byteBuf.toString(CharsetUtil.UTF_8)); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {/** * @author Jiang Chengjun * @date 2020/6/1 11:20 * @description Specifies an I/O event Exception **/ cause.printStackTrace(); ctx.close(); }}Copy the code

Code description:

  • @channelHandler. Sharable, this annotation is for thread safety, if you don’t care about thread safety, don’t add it.

  • SimpleChannelInboundHandler, the type can be ByteBuf, also can be a String, could also be an object, according to the actual situation.

  • ChannelRead0, the message reading method, notice the 0 in the name.

  • ChannelHandlerContext, Channel context.

  • ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf, ByteBuf

  • Charsetutil.utf_8, a JDK native method that specifies the encoding format for converting byte arrays to strings.

Create a client startup class

The client startup class establishes a connection according to the IP address and port of the server. After the connection is established, the bidirectional transmission of messages is realized.

The code is simple as follows:

package com.jcj.helloworld; import com.sun.org.apache.bcel.internal.generic.ATHROW; import io.netty.bootstrap.Bootstrap; import io.netty.buffer.Unpooled; import io.netty.channel.ChannelFuture; import io.netty.channel.ChannelInitializer; import io.netty.channel.EventLoopGroup; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.SocketChannel; import io.netty.channel.socket.nio.NioSocketChannel; import io.netty.util.CharsetUtil; import java.net.InetSocketAddress; /** * @auther: @date: 2020/6/1 11:24 * @description: Public class AppClientHello {private Final String host; private final int port; public AppClientHello(String host, int port) { this.host = host; this.port = port; } public void run() throws Exception {/** * @author Jiang Chenjun * @date 2020/6/1 11:28 * @description Sets corresponding parameters. **/ EventLoopGroup group = new NioEventLoopGroup(); //I/O thread pool try {Bootstrap bs = new Bootstrap(); Bs.group (group).channel(niosocketchannel.class) instantiates a channel.remoteAddress (new) InetSocketAddress(host,port)).handler(new ChannelInitializer<SocketChannel>()// Initialize the channel. {@override protected void initChannel(SocketChannel socketChannel) throws Exception { socketChannel.pipeline().addLast(new HandlerClientHello()); // add our custom Handler}}); // Connect to the remote node; ChannelFuture future=bs.connect().sync(); Unpooled.copiedbuffer (utF-8 future.channel().writeAndFlush(unpooled.copiedBuffer ())"Hello World", CharsetUtil.UTF_8)); CloseFuture () opens a listener for a channel(during which time the channel is working) until the link disconnects future.channel().closeFuture().sync(); } finally { group.shutdownGracefully().sync(); } } public static void main(String[] args) throws Exception { new AppClientHello("127.0.0.1",18080).run(); }}Copy the code

Since the code has been extensively commented, only a few examples are described here:

  • ChannelInitializer is used to initialize a Channel, such as adding multiple handlers.

  • Bs.connect ().sync(), where sync() indicates the synchronization method used, so that the connection is successfully established before continuing.

  • Pipeline (), the connection is established, will automatically create a pipeline pipeline, this pipeline is also known as the chain of responsibility, ensure the sequence of execution, but also can flexibly configure all kinds of Handler, this is a very subtle design, not only reduce the resource overhead brought by thread switching, avoid a lot of trouble, At the same time, performance has been greatly enhanced.

4.2 Creating a Server Class

Create a Handler

As with the client, only the message read method channelRead(note this is not channelRead0) and the exceptionCaught method are overridden.

The other server-side Handler is inherited ChannelInboundHandlerAdapter, rather than SimpleChannelInboundHandler, as for the difference, don’t go into here, everyone should baidu.

The code is as follows:

package com.jcj.helloworld; import io.netty.buffer.ByteBuf; import io.netty.buffer.Unpooled; import io.netty.channel.ChannelHandler; import io.netty.channel.ChannelHandlerContext; import io.netty.channel.ChannelInboundHandlerAdapter; import io.netty.util.CharsetUtil; /** * @auther: @date: 2020/6/1 11:47 * @description: The server I/O processing class. * / @ ChannelHandler Sharable public class HandlerServerHello extends ChannelInboundHandlerAdapter {@ Override Public void channelRead(ChannelHandlerContext CTX, Object MSG) throws Exception {// Processes received data and sends the message to the client ByteBufin = (ByteBuf) msg;
        System.out.println("Received a message from client:"+ in.toString(CharsetUtil.UTF_8)); // Write and send information to remote (client) ctx.writeAndFlush(Unpooled. CopiedBuffer ("Hello, this is the server. I have received your message.", CharsetUtil.UTF_8)); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {printStackTrace(); Throwable cause) throws Exception {printStackTrace(); ctx.close(); }}Copy the code

The above code is very succinct, notice the comparison with the client Handler class.

Create a server-side startup class

The server side startup class is a bit more complicated than the client side startup class. First post the following code:

package com.jcj.helloworld; import io.netty.bootstrap.ServerBootstrap; import io.netty.channel.ChannelFuture; import io.netty.channel.ChannelInitializer; import io.netty.channel.EventLoopGroup; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.SocketChannel; import io.netty.channel.socket.nio.NioServerSocketChannel; import java.net.InetSocketAddress; /** * @auther: @date: 2020/6/1 11:51 * @description: Public class AppServerHello {private int port; public AppServerHello(int port) { this.port = port; } public void run() throws Exception { EventLoopGroup group = new NioEventLoopGroup(); //Netty's Reactor thread pool initializes a NioEventLoop array for I/O operations such as accepting new connections and reading/writing data. Try {ServerBootstrap b = new ServerBootstrap(); / / used to launch the NIO service b.g roup (group). The channel (NioServerSocketChannel. Class) / / through a channel factory method design pattern instantiation, localAddress (new InetSocketAddress(port))// Set the listening port. ChildHandler (new ChannelInitializer<SocketChannel>() {//ChannelInitializer is a special handler class, Override public void initChannel(SocketChannel CH) throws Exception. Override public void initChannel(SocketChannel CH) throws Exception {//ChannelInitializer is a special handler class that helps users configure a new Channel. ch.pipeline().addLast(new HandlerServerHello()); // Configure childHandler to notify an instance of InfoServerHandler about message handling}}); ChannelFuture ChannelFuture = b.bind().sync(); ChannelFuture = b.bind().sync(); System.out.println("In" + channelFuture.channel().localAddress()+"On tap."); // Block, closeFuture() opens a listener for a channel(during which the channel is working) until the link disconnects. } finally { group.shutdownGracefully().sync(); // Close EventLoopGroup and release all resources, Public static void main(String[] args) throws Exception {new AppServerHello(18080).run(); }}Copy the code

Code description:

  • In the actual project, two instances of the EventLoopGroup are created, one for receiving client connections and the other for handling message I/O. To simplify the process, one instance is used to do both.

  • NioServerSocketChannel, instantiate a channel through a factory through the factory method design pattern, this is not a need to go into the case of Netty project development is not skilled.

Now that we have written both the server and the client, how to Run it? First right-click on the server startup class and click Run ‘AppServerHello.main()’ to Run it.

Then, do the same thing, run the client startup class, and you’ll see the effect.

5, the end

That concludes this article, which hopefully gives you an overview of Netty and its development process.