preface
The original purpose of the Internet was to transmit text (text conversations). So how does the page appear to us after we send the request using the browser? The next step is to introduce the process from the URL to the rendering page by using the image.
Text dialogue — from request to response
(1) The browser sends a Request to the server where the website is located.
(2) The website server processes and parses the Request after receiving it
(3) Then return a Response (Response) to the browser, which contains the source code of the page and other content
(4) The browser then parses it and presents the web page.
What rules is the process of this text dialogue based on? In short, this communication process is based on TCP/IP communication protocol family specification, complete a series of information exchange process from the client to the server.
Introduction to TCP/IP protocol family
What is the TCP/IP protocol family?
The purpose of TCP/IP protocol family is to establish rules to enable the exchange of information between computers.
The communication between the two parties must be based on the same method. For example, the rules such as which side initiates the communication first, which language to use, and how to end the communication need to be determined in advance. We call these rules protocols. Usually we say TCP/IP protocol family is the general name of all kinds of protocol family related to the Internet.
Here is the important hierarchical division of TCP/IP, which can be divided into four layers: application layer, transport layer, network layer and link layer. (The advantage of layering is that the design inside each layer can be changed freely and linked through the interfaces of each layer, whereas if there is only one protocol, all the involved parts need to be redesigned.)
2, the function of TCP/IP layer
(1) Application layer: determines the communication activities when providing application services to users. The application layer is responsible for transmitting various final forms of data and is the layer that directly deals with users. Typical protocols are HTTP and FTP.
(2) Transport layer: responsible for transmitting text data. The transport layer has two different protocols: Transmission Control Protocol (TCP) and User Data Protocol (UDP).
(3) Network layer: responsible for the allocation of addresses and transmission of binary data, the main protocol is IP protocol;
(4) Link layer: responsible for the establishment of circuit connections, is the physical basis of the entire network, typical protocols including Ethernet, ADSL, etc.
3, TCP/IP communication transmission flow
How does data flow between the functional layers of TCP/IP?
(1) The client as the sender sends an HTTP request (for example, to browse www.baidu.com) at the application layer (using THE HTTP protocol) and generates an HTTP packet.
(2) To facilitate transmission, the transport layer (TCP protocol) divides the data (HTTP request packets) received from the application layer, marks the serial number and port number of each packet, and forwards the packets to the network layer.
(3) At the network layer (IP protocol), add the MAC address as the communication destination and forward it to the link layer.
(4) Attach an Ethernet header to the data and send it. The generated Ethernet data packet will be transmitted to the receiver through the physical layer.
(5) The server at the receiving end receives data at the link layer and sends the data to the upper layer in sequence until it reaches the application layer. The HTTP request sent by the client is truly received only when it is transmitted to the application layer.
In the communication process, a header information of the layer must be stamped when each layer passes through. On the contrary, when the receiver transmits data between layers, the corresponding head will be eliminated after each layer.
Third, based on TCP/IP communication process
A diagram illustrating the communication process from request to web rendering (the following diagram is based on IP protocol, TCP protocol, DNS service and HTTP protocol) and explaining each step:
1. The browser enters the URL to send the request
The Uniform Resource Locator (URL) is a Web page address that needs to be entered when a Web browser is used to access a Web page.
A URL consists of the following elements:
(2) Hierarchical URL token: “//” is fixed
(3) Login information: credential information required for accessing resources (omitted)
(4) Server address: Usually a domain name, sometimes an IP address (in actual communication, the IP address is required, and the domain name is resolved by the DNS server)
(5) Port number: Indicates the port number in numeric mode. If the port number is the default value of HTTP, “:80” can be omitted
(6) Path: Use “/” to distinguish each directory name in the path
(7) Query: form parameters of GET mode, with “?” Each parameter is separated by ampersand (&), and the parameter name and data are separated by =. Usually, the URL encoding is UTF8 to avoid character conflicts
(8) Fragment: Starting with the “#” character, the fragment identifier can usually be used to mark the sub-resources in the obtained resource
2. The DNS resolves the URL domain name in the request
Computers can be assigned both IP addresses, host names, and domain names. Users usually use host names or domain names to access each other’s computers, rather than IP addresses. It was easier for a computer to process a set of numbers, so DNS was born. The DNS protocol provides the service of looking up an IP address by a domain name (or reversely looking up a domain name from an IP address).
3. HTTP protocol generates request packets
HTTP: HyperText Transfer Protocol Located at the application layer, the HyperText Transfer Protocol determines a series of communication contents and modes from the client to the server. The communication is completed by generating and sending packets.
4. TCP protocol provides reliable byte stream transmission service
TCP: Transmission Control Protocol. It is located at the transport layer.
(1) Byte Stream Service: To facilitate transmission, chunk data is divided into data packets in the unit of segment for management.
(2) Reliable transmission service refers to the ability to transmit data to each other accurately and reliably. TCP adopts strategies such as three-way handshake connection to ensure the reliability of transmission (three-way handshake, four-way wave will be emphasized at the end of the article).
5, IP protocol to achieve data transfer to the other side of the computer
Internet Protocol (IP) Is located at the network layer. The function of IP protocol is to realize the packet to the IP address of the other party’s computer. The communication between IP addresses depends on the MAC address (the fixed address of the network adapter). You need to use ARP to check the MAC address of the communication party.
6. Receive and parse the request packet and send back the response packet
4. Establishing and disconnecting TCP Connections (key supplement)
Establishing a TCP connection (3-way Handshake)
TCP provides connection-oriented communication. Connection-oriented communication means that preparations are made between the two ends before data communication begins. A three-way handshake is a TCP connection that requires the client and server to send a total of three token packets to confirm the connection. Here’s a flow chart of the three handshakes:
First handshake: The client sets the SYN flag bit to 1, generates a value seq=J randomly, and sends the packet to the server. The client enters the SYN_SENT state, waiting for the server to confirm.
Second handshake: After the server receives the packet, the flag bit SYN=1 knows that the client requests to establish a connection. The server sets the flag bit SYN and ACK to 1, ACK =J+1, randomly generates a value seq=K, and sends the packet to the client to confirm the connection request, and the server enters the SYN_RCVD state.
The third handshake: After receiving the acknowledgement, the client checks whether the ACK is J+1 and ACK is 1. If it is correct, it sets the flag bit ACK to 1 and ACK =K+1 and sends the data packet to the server. The server checks whether the ACK is K+1 and ACK is 1. The client and server enter the ESTABLISHED state and complete the three-way handshake to establish the connection. Then data can be transmitted between the client and server.
Why 3-way handshakes: The first two handshakes are obviously necessary, but the last one, when the client receives the acknowledgement from the server, why would it want the server to send another acknowledgement? This is mainly to prevent the invalid request segment suddenly sent to the server and the connection misjudgment.
Consider the following situation: The client sends a connection request segment to the server, but the segment is delayed on some network nodes for a long time. Therefore, the client sends another connection request segment to the server due to timeout. Then the connection is established, data transmission is complete, and the connection is released. If for the first time send a request message segment (expired) delayed after a period of time, then the service side, obviously, this is a failure message segment already, but after you receive the service side will mistakenly assume that the client again in a connection request, and send a confirmation message to the client, and agreed to establish a connection. Assuming the three-way handshake is not used, the server simply sends an acknowledgement and a new connection is established. But since the client hasn’t issued to establish a connection request, so you don’t ignore server-side validation, also won’t send data to the server, and the server is thought that a new connection has been established, and has been waiting for the client to send data, so the server will have been waiting for, until beyond the set value of keep alive the counter, and that it would be wrong for the client to determine, To close the connection. This wastes a lot of server resources. If the three-way handshake is used, the client does not send acknowledgement to the server, and the server, because it does not receive acknowledgement, knows that the client did not request a connection and does not establish the connection.
TCP Disconnection (four waves)
TCP connections are full-duplex. Therefore, each direction must be closed separately. To terminate a TCP connection, the client and server must send a total of four packets to confirm the disconnection. Here’s a diagram of the four waves:
Note: The disconnecting end can be either the client or the server. The following example is an interrupt request from a client.
First wave: The client sends a FIN=M to stop data transfer from the client to the server. The client enters the FIN_WAIT_1 state. This means “my client has no data to send you”, but if you still have data to send from the server, you don’t have to close the connection. You can continue sending data.
Second wave: After receiving the FIN, the server sends ACK =M+1, telling the client, “I received your request, but I’m not ready yet, please continue waiting for my message.” Then the client enters the FIN_WAIT_2 state and continues to wait for FIN packets from the server.
Third wave: When the server confirms that the data has been sent, it sends a FIN=N packet to the client, telling the client that it is ready to close the connection. The server enters the LAST_ACK state.
The fourth wave: After receiving the FIN=N packet, the client knows that it can close the connection, but it still does not trust the network. Therefore, the client sends ACK=1, ACK= N+1 and enters TIME_WAIT state. If the server does not receive an ACK, the client can retransmit the packet. When the server receives the ACK, it knows it is ready to disconnect (CLOSED state). If the client does not receive a reply after 2MSL (MSL is called the maximum packet life, and the RFC recommends setting the MSL to 2 minutes), the server is shut down and the client can close the connection. Four handshakes were completed.
Why four wave: TCP is a connection-oriented, reliable byte stream transport layer protocol, TCP is full duplex mode, this means that when the client sends a FIN message segment, saying only that the client has no data to be sent, the client tell the server, it’s all data has been sent out; However, the client can still accept data from the server; When the server returns an ACK packet segment, it indicates that it knows that the client has no data to send, but host 2 can still send data to the client. When the server also sends the FIN segment, it means that the server has no data to send either, and it tells the client that it has no data to send either. After that, the TCP connection will be pleasantly interrupted.
Why the client waits for 2MSL in TIME_WAIT mode: (1) To ensure that the last ACK segment sent by the client can reach the server. The ACK segment is likely to be lost, so that the server in the list-ACK state cannot receive the acknowledgement of the sent FIN+ACK segment. The server may retransmit the FIN+ACK segment, and the client will receive the retransmitted FIN+ACK segment within the 2MSL time. Then the client retransmits a confirmation, restarts the 2MSL timer, and both the client and server enter the CLOSED state. (2) Prevent invalid request connection from appearing in this connection. When connected in MSL 2 wait, any late message will be discarded, because in MSL 2 wait, by the socket (socket means of IP and port, socket) define the connection during this period will not be used again, so you can make the next new connections will not appear this kind of delay before the old connection message segment.
summary
Above, we understand the role of TCP/IP protocol and communication process, for THE Http protocol will be introduced in detail later.
References:
Illustrated HTTP (download address)
The TCP/IP Protocol (Network Protocol 2)
Ruan Yifeng, TCP-IP Model
Setting up and Releasing TCP Connections