Preface: As a developer, we often hear the words HTTP protocol, TCP/IP protocol, UDP protocol, Socket, Socket long connection, Socket connection pool and so on. However, the relationship, difference and principle between them are not clear to everyone. This article starts from the network protocol foundation to Socket connection pool. Explain their relationship step by step.

Seven-layer network model

Start with the layered model of network communication: the seven-layer model, also known as the Open System Interconnection (OSI) model. From bottom to top, it consists of the physical layer, data link layer, network layer, transport layer, session layer, presentation layer and application layer. Everything about communication depends on it, and the following picture shows some of the protocols and hardware for each layer.

From the above figure, I know that IP protocol corresponds to the network layer, TCP and UDP protocol corresponds to the transport layer, and HTTP protocol corresponds to the application layer. OSI does not have sockets. Then what is Socket?

TCP and UDP connections

On the transmission layer TCP, UDP protocol may we usually meet more, some people say TCP is secure, UDP is not secure, UDP transmission is faster than TCP, then why, we start from the TCP connection to establish the process of analysis, and then explain the difference between UDP and TCP.

TCP’s three handshakes and four breakups

We know that TCP requires three handshakes to establish a connection and four breakups to disconnect a connection, and what and how those three handshakes and four breakups are done.

First handshake

Second handshake: The server should acknowledge the SYN segment received from the client, and set this Acknowledgment Number to X +1(Sequence Number+1). Set the SYN position to 1 and Sequence Number to y. The server puts all the above information into a packet segment (namely, SYN+ACK packet segment) and sends it to the client. At this time, the server enters the SYN_RECV state.

Third handshake: The client receives a SYN+ACK packet from the server. A. Then set this Acknowledgment Number to Y +1 and send an ACK segment to the server. After this segment is sent, both the client and the server enter the ESTABLISHED state and complete the TCP three-way handshake.

Once the three-way handshake is complete, the client and server can begin transferring data. So that’s the overview of the TCP three-way handshake. When the communication ends, the client and server are disconnected, and four breakups are required for confirmation.

First break up: Host 1 (either client or server), set the Sequence Number and Acknowledgment Number, and send a FIN segment to host 2. At this point, host 1 enters the FIN_WAIT_1 state. This means that host 1 has no data to send to host 2.

Second break off: Host 2 receives the FIN segment from host 1 and sends an ACK segment back to Host 1. This Acknowledgment Number is set to Sequence Number plus 1. Host 1 enters the FIN_WAIT_2 state. Host 2 tells host 1 that I “agree” to your shutdown request;

Third breakup: Host 2 sends a FIN packet to host 1 to close the connection, and host 2 enters the LAST_ACK state.

For the fourth time, host 1 receives the FIN packet from host 2 and sends an ACK packet to host 2. Then host 1 enters the TIME_WAIT state. Host 2 closes the connection after receiving the ACK packet from host 1. If host 1 does not receive a reply after waiting for 2MSL, then the Server is shut down.

It can be seen that a TCP request is established and closed at least seven times, which does not cover data communication, while UDP does not require three handshakes and four breakups.

Differences between TCP and UDP

  1. TCP is link-oriented, although the insecure and unstable characteristics of the network determine how many times the handshake can not ensure the reliability of the connection, but TCP three handshake in the minimum (in fact, also to a large extent) to ensure the reliability of the connection; UDP is not connection-oriented. It does not establish a connection with the other party before transmitting data, and does not send an acknowledgement signal to the received data. The sender does not know whether the data will be correctly received and does not need to resend the data.

  2. It is also because of the features mentioned in 1 that UDP has lower overhead and higher data transmission rate. Because there is no need to confirm sending and receiving data, UDP has better real-time performance. Know the difference between TCP and UDP, it is not hard to understand why the TCP transport protocol is slower than the transfer files using UDP QQ MSN, but doesn’t say QQ communication is not secure, because programmers can manually to validate the UDP data transceiver, such as the sender of each packet number and then validated by the receiving party ah of what, Even so, UDP achieves the transmission efficiency that TCP cannot achieve because it does not adopt the “three-way handshake” similar to TCP in the encapsulation of the underlying protocol.

The problem

We often hear questions about the transport layer

1. What is the maximum number of concurrent connections to the TCP server?

A common misconception about the maximum number of concurrent connections on a TCP server is that “because the port number is capped at 65535, a TCP server can theoretically support 65535 concurrent connections.” You need to understand the components of a TCP connection: client IP address, client port, server IP address, and server port. Therefore, for a TCP server process, the number of clients it can connect to at the same time is not limited by the number of available ports. In theory, the number of connections a port on a server can establish is the number of IP addresses in the world * the number of ports per machine. The actual number of concurrent connections is limited by the number of files Linux can open, which is configurable and can be very large, so it is really limited by system performance. Use #ulimit -n to check the maximum number of file handles for the service. Use ulimit -n XXX to change the number of file handles you want to open. You can also modify system parameters:

2. Why does TIME_WAIT wait 2MSL before returning to CLOSED?

This is because although both parties have agreed to close the connection and all four packets have been coordinated and sent, they can logically return to the CLOSED state (just like the SYN_SEND state to the ESTABLISH state). However, assume that the network is unreliable and you cannot guarantee that the ACK packet you send will be received by the peer. Therefore, the peer Socket in the LAST_ACK state may resend the FIN packet because it fails to receive the ACK packet due to timeout. Therefore, the TIME_WAIT state is used to resend ACK packets that may be lost.

3. What is the problem if the TIME_WAIT state needs to wait 2MSL before returning to CLOSED

After a TCP connection is established between the communication parties, the party who actively closes the connection enters the TIME_WAIT state. The TIME_WAIT state lasts for two MSLS, that is, 1-4 minutes, or 4 minutes in the Windows operating system. A connection in TIME_WAIT state usually occupies a local port for a client. The maximum number of the upper end of a machine is 65536. If the pressure test is conducted on the same machine to simulate tens of thousands of customer requests, and the short connection communication is conducted with the server, then the machine will generate about 4000 TIME_WAIT sockets. If Nginx is used as the direction proxy, you also need to consider the TIME_WAIT state. If a large number of TIME_WAIT state connections exist in the system, you can adjust kernel parameters to solve the problem.

Then run /sbin/sysctl -p for the parameters to take effect.

Net.ipv4. tcp_syncookies = 1 SYN Cookies are enabled. When SYN overflow occurs, cookies are enabled to prevent a small number of SYN attacks. The default value is 0, indicating that the SYN wait queue is disabled.

Net.ipv4. tcp_tw_reuse = 1: enables reuse. Allow time-Wait Sockets to be re-used for new TCP connections. Default is 0, indicating closure.

Net.ipv4. tcp_TW_recycle = 1 Enables the fast recycling of time-wait Sockets in TCP connections. The default value is 0, indicating that the fast recycling of time-Wait sockets is disabled.

Net.ipv4. tcp_fin_timeout Changes the default TIMEOUT period.

The HTTP protocol

The network has an easy to understand introduction to TCP/IP and HTTP: “We could just use [transport layer] TCP/IP when transferring data, but then without the application layer, the content of the data would not be recognized. Application layer protocols must be used if you want to make sense of the data being transferred. There are many application-layer protocols, such as HTTP, FTP, and TELNET. You can also define application-layer protocols.

HTTP, also known as Hypertext Transfer Protocol (Hypertext Transfer Protocol), is the foundation of Web networking and one of the commonly used protocols for mobile networking. The Web uses HTTP as an application layer Protocol to encapsulate HTTP text information. Then use TCP/IP as the transport layer protocol to send it to the network.

Because HTTP actively releases the connection at the end of each request, an HTTP connection is a “short connection” that requires continuous connection requests to the server to keep the client program online. The usual practice is that even though no data is needed, the client keeps sending a “stay connected” request to the server at regular intervals, and the server replies to the client after receiving the request, indicating that it knows the client is “online.” If the server cannot receive requests from the client for a long time, the client is considered offline. If the server cannot receive any reply from the server for a long time, the network is disconnected.

Here is a simple REQUEST for HTTP Post Application/JSON data content:

About Sockets

We now understand that TCP/IP is just a stack of protocols that, like an operating system, must be implemented and provided with an external interface. Just as the operating system provides a standard programming interface, such as the Win32 programming interface, TCP/IP must also provide a programming interface to the outside world, which is called Socket. Now we know that sockets are not necessarily related to TCP/IP. When Socket programming interface is designed, it is hoped that it can also adapt to other network protocols. Therefore, the emergence of Socket is more convenient to use the TCP/IP protocol stack, which abstracts TCP/IP, forming a few of the most basic function interface. Examples include Create, Listen, Accept, connect, read, and write.

Each language has its own library for creating a Socket server and client. Here is an example of how Nodejs creates a server and client:

Server:

The service listens on port 9000

The following uses the command line to send HTTP requests and Telnet

Note that curl processes the packet only once.

The client

Long Socket connection

A long connection means that multiple packets can be sent continuously on a TCP connection. During the TCP connection, if no packets are sent, both parties need to send detection packets (heartbeat packets) to maintain the connection. In general, the two parties need to maintain the connection online. A short connection is a TCP connection established when data is exchanged between the communication parties. After data is sent, the TCP connection is disconnected. For example, Http, only connection, request, close, the process time is short, if the server does not receive a request within a period of time can close the connection. In fact, long connection is relative to the usual short connection, that is, keep the client and the server connected for a long time.

The usual short connection operation steps are:

Connection → data transmission → close connection;

And long links are usually:

Connect → Data Transfer → Hold the connection (heartbeat)→ Data transfer → Hold the connection (heartbeat)→… → Close the connection;

When to use long connection, short connection?

Long connections are used for frequent, point-to-point communication, and the number of connections should not be too many. Each TCP connection requires a three-step handshake, which takes time. If each operation is connected first and then operated, the processing speed will be much lower. Therefore, after each operation is finished, it is OK to send packets directly, and there is no need to establish TCP connection. For example, if the database is connected to a long connection, frequent communication with a short connection will cause Socket errors, and frequent Socket creation is also a waste of resources.

What is a heartbeat pack and why is it needed:

A heartbeat packet is a self-defined command that periodically notifies the client and server of its status. It is sent at a certain interval and is similar to a heartbeat. Therefore, it is called a heartbeat packet. Socket is used to receive and send data in the network. However, if the socket is already disconnected (for example, if one party is disconnected), there are bound to be problems sending and receiving data. But how do you know if this socket is still usable? This requires the creation of a heartbeat mechanism in the system. In fact, TCP already implements a mechanism for us called heartbeat. If you set the heartbeat, TCP will send the number of heartbeats you set (say 2) within a certain amount of time (say 3 seconds), and this information will not affect the protocol you define. It can also be defined by itself. The so-called “heartbeat” is to periodically send a customized structure (heartbeat packet or heartbeat frame) to let the other party know that they are “online” to ensure the validity of the link.

Implementation:

Server:

Server output:

Client code:

Define your own protocol

Application layer protocols such as Http, Mqtt, Dubbo, and so on must be used if the data being transferred is to be meaningful. The following problems need to be solved in the protocol of user-defined application layer based on TCP:

  1. Definition and processing of heartbeat packet format

  2. The definition of a header is that when you send data you need to send a header, and the header will parse out the length of the data that you want to send

  3. The format in which you send the packet, whether it’s JSON or some other serialization

Let’s define our own protocol and write the service and client to call:

Define the header format: Length: 000000000XXXX; XXXX represents the length of data. The total length is 20.

Data serialization: JSON.

Server:

The client

Log printing:

Here you can see a client in the same time processing a request can be a very good job, but imagine such a scene, if the same time for the same client to invoke server-side request for many times, send head data and content for many times, the data from the server event data are difficult to distinguish which data is received which requests, For example, if two headers arrive at the server at the same time, the server will ignore one of them, and the subsequent content data may not correspond to the header. So if you want to reuse long connections and be able to handle server requests with high concurrency, you need connection pooling.

The Socket connection pool

What is a Socket connection pool? A pool is a collection of resources. Therefore, a Socket connection pool is a collection that maintains a certain number of Socket connections. It can automatically detect the validity of Socket long connections, eliminate invalid connections, and supplement the number of long connections in the connection pool. At the code level, this is an artificial class that implements this function. Generally, a connection pool contains the following properties:

  1. Idle and usable long connection queues

  2. A long connection queue for running communication

  3. A queue of requests waiting to get an idle long connection

  4. Rejection of invalid long connections

  5. Specifies the number of long-link resource pools

  6. Creation of long-link resources

Scenario: A request is sent to the resource pool to obtain a long-connection resource. If there is a long-connection in the idle queue, the Socket is obtained and moved to the running long-connection queue. If there are no idle queues and the number of running queues is smaller than the configured number of resources in the connection pool, a new long connection is created to the running queue. If the number of running queues is equal to the configured number of resources in the connection pool, the request goes to the waiting queue. When a running Socket completes the request, it moves from the running queue to the idle queue and triggers the waiting request queue to obtain the idle resource, if any.

Here’s a brief introduction to node.js’s generic-pool module: generic-pool.

Main file directory structure

Initialize the connection pool

Using connection pools

The following connection pool usage, using the protocol is our previous custom protocol.

Socket_pool 127.0.0.1 9000 connect: No new Socket connection is established for the first two requests.

Source code analysis

The main code is found in the pool.js constructor in the lib folder: lib/ pool.js

You can see that there are idle resource queues, requesting resource queues, waiting request queues, and so on.

Now look at the pool.acquire method

lib/Pool.js

The above code follows this pattern until it finally gets the resource of the long connection, and you can learn more about it yourself.

About the author: Changfeng, 6 years of experience in server-side development, responsible for the evolution and development deployment of technical solutions from zero to high concurrent access of start-up projects, accumulated certain experience in platform development and high concurrent and high availability, and now responsible for the architecture and development of the gateway layer under the company’s micro-service framework.