preface

It has been three years since THE author opened the front end of the advanced way, so far there is also 17K star, thanks to the readers love. At the moment, some of the content is a little outdated, so I decided to rewrite the content.

The revamped content will be assembled in a “dry explosion front” for interested readers to check out.

Important before Reading:

This article is not an encyclopaedia. It is only designed to prepare for the interview review, to fill in the gaps, to get into the introduction of a knowledge point, to understand the relevant interview questions, etc.

I have always been advocating to learn the knowledge points involved in the interview questions, rather than brush a lot of interview questions, the result will not change a question type. Therefore, this article is different from other surface, aiming at refining the common knowledge points under the interview questions, rather than dumping a lot of interview questions to you.

You can also read on the author’s website, better experience!

17K STAR warehouse, solve 90% of the basic interview questions of large factories

Today’s article starts by typing in the URL and talking about what front-end engineers need to know about networking.

Note: there is a lot of network knowledge involved in the process of entering the URL into the returned data. I will only introduce what front-end engineers need to know. The rest is for readers to learn by themselves.

When we type the URL: www.google.com/ into a browser, the browser first looks for the IP address of the domain name. After all, we still have to use IP to find the address of the other party. The domain name is just a convenient alias.

So how to find the IP address corresponding to the domain name? Next, let me introduce you to the first piece of content we encountered: DNS.

DNS

DNS is used to query specific IP addresses through domain names.

Domain names were invented because the combination of numbers and English (IPv6) of IP addresses was not good for human memory. You can think of a domain name as an alias for an IP address, and DNS is to find out what the real name of that alias is.

When you want to access www.google.com in your browser, do the following:

  1. The local client sends a request to the server to query the IP address
  2. Check whether the browser has an IP address cache for the domain name
  3. Check whether the IP address of the domain name is cached in the operating system
  4. Check whether the Host file has the resolution configuration for the domain name
  5. If not, go directly to the DNS root server to query, this step will find the responsiblecomThe server for this level 1 domain name
  6. Then go to the server to querygoogle.comThis secondary domain name
  7. Next querywww.google.comThe address of this level 3 domain name
  8. Return to the DNS client and cache it

DNS recursive query is introduced above, there is a kind of iterative query, the difference is that the former is the DNS server configured by the system to do the request, the results of the data returned to the client; The latter is requested by the client.

Curious readers may ask: Is the REQUEST sent by DNS to these servers based on TCP or UDP protocol?

In fact, in the current network environment, both protocols are useful. Through UDP to carry out some small data requests, this time can use the advantages of UDP fast performance; TCP is used to request large amounts of data to ensure the integrity and order of the data to ensure the correctness and integrity of the data.

We will also cover UDP and TCP in a future article.

After obtaining the IP address of the domain name, the browser immediately establishes a connection with the server through TCP. Let’s talk about TCP.

TCP

TCP requires a three-way handshake to establish a connection:

Both ends are initially CLOSED. Before communication begins, both parties create a TCB (a data structure that contains much of what the protocol requires, if you are interested). After the TCB is created, the server enters the LISTEN state and waits for the client to send data.

First handshake

The client sends a connection request segment (SYN) to the server. The paragraph contains its own initial data communication sequence number. After the request is SENT, the client enters the SYN-sent state. X indicates the initial number of data communication on the client.

Second handshake

After receiving a connection request packet and agreeing to the connection, the server sends an ACK response (ACK + SYN) containing its initial data communication sequence number. After the response is complete, the server enters the SYN-received state.

Third handshake

When the client receives a connection approval reply, it sends an ACK to the server. After sending the packet, the client enters the ESTABLISHED state. After receiving the response, the server also enters the ESTABLISHED state.

PS: The third handshake can contain data, via TCP Quick Open (TFO) technology. In fact, as long as the handshake protocol is involved, TFO can be used, the client and the server store the same cookie, the next handshake cookie to reduce the RTT (one request back and forth time).

Here’s a classic interview question: Why does TCP need three handshakes instead of two?

This prevents the invalid connection request segment from being received by the server, resulting in an error.

Imagine the following scenario. The client sends A connection request A but times out due to network reasons. In this case, TCP enables the timeout retransmission mechanism to send A connection request B again. At this point, the request smoothly arrives at the server, and the server responds and establishes the request. If connection request A finally reaches the server after both ends close, the server thinks that the client needs to establish A TCP connection again, and responds to the request and enters the ESTABLISHED state. In this case, the client is actually in the CLOSED state, which causes the server to wait all the time, resulting in a waste of resources.

PS: If either end of the connection is disconnected, TCP resends SYN packets and tries five times. SYN FLOOD attacks may occur during the connection establishment. In this case you can either lower the number of retries or simply reject the request if it cannot be processed.

When a connection is established and the request is completed, the TCP connection is not immediately broken. Thanks to the KEEP-alive attribute in HTTP 1.1, the connection can be kept for a short period of time. The specific disconnection depends on the Settings of the server or the client.

Once close, TCP disconnects the connection with a four-way handshake.

First handshake

If client A considers the data transfer complete, it needs to send A connection release request to server B.

Second handshake

After receiving the connection release request, USER B tells the application layer to release the TCP connection. The ACK packet is then sent and the state CLOSE_WAIT is entered, indicating that the connection from A to B has been released and the data sent by A will not be received. But because the TCP connection is bidirectional, B can still send data to A.

Third handshake

If there is still uncompleted data sent by USER B, user B will send A connection release request to user A. Then, user B enters the last-ACK state.

PS: The second and third handshakes of the server can be combined to delay the sending of the ACK packet through the technique of delayed acknowledgement (usually with a time limit, otherwise the other party will mistakenly think that a retransmission is needed).

The fourth handshake

After receiving the release request, user A sends A confirmation reply to user B. User A enters the time-wait state. The state lasts for 2MSL (maximum segment lifetime, which refers to the duration of the packet segment in the network. The timeout will be discarded). If there is no resending request from B within this period, the state is CLOSED. When B receives the confirmation reply, it also enters the CLOSED state.

Why does A enter the time-wait state and WAIT 2MSL before entering the CLOSED state?

In order to ensure that B can receive A’s confirmation. If A enters the CLOSED state directly after sending the confirmation reply, if the confirmation reply does not arrive due to network problems, B cannot be CLOSED normally.

Both types of handshakes involved in establishing and disconnecting connections are important knowledge in TCP. Of course, there are a lot of things we need to learn about TCP, such as how the TCP protocol achieves orderly and complete transmission of data, which we will learn in a moment.

ARQ protocol

First, let’s learn how TCP implements complete delivery of data. After all, network fluctuations and various uncertainties will lead to the loss of data transmission. Once this happens, we have to ensure that the protocol has a retransmission mechanism. Just like when we upload some important data in a business, when we don’t get a response once or something else happens, we retry multiple times.

ARQ protocol is also known as timeout retransmission mechanism protocol. The correct delivery of data is guaranteed by acknowledgement and timeout mechanism, including stop waiting ARQ and continuous ARQ protocol.

Stop waiting for ARQ

Normal transmission process

Whenever A sends A packet to B, it stops sending and starts A timer to wait for the reply from the peer end. After receiving the reply from the peer end within the timer time, it cancels the timer and sends the next packet.

When something goes wrong

1. Packets are lost or an error occurs

Packet loss may occur during packet transmission. When the timer expires, the packet loss data will be sent again until the peer end responds. Therefore, back up the sent data every time.

Even if the packet is normally transmitted to the peer end, packet errors may occur during transmission. In this case, the peer end discards the packet and waits for the retransmission.

PS: Generally, the timer setting time is greater than the average time of an RTT.

2. The ACK times out or is lost

The reply transmitted by the peer end may also be lost or timed out. If the timer expires, end A retransmits packets. When receiving A packet with the same serial number, end B discards the packet and replies until end A sends the next packet with the same serial number.

In the case of timeout, A terminal will determine whether the serial number has been received. If so, it only needs to discard the reply.

The disadvantage of this protocol is that the transmission efficiency is low. In a good network environment, each packet must wait for the ACK of the peer end.

Continuous ARQ

In continuous ARQ, the sender has a sending window and can continuously send the data in the window without receiving any reply. In this way, the waiting time is reduced and the efficiency is improved compared with stopping the waiting ARQ protocol.

The cumulative confirmation

In continuous ARQ, the receiver continuously receives packets. It would be a waste of resources if it stopped waiting for a packet to be received in ARQ and then sent a reply. After multiple packets are received, a reply packet can be sent. The ACK in the packet can be used to tell the sender that all the data before the sequence number has been received. Please send the data with the sequence number + 1 next time.

But there is a downside to cumulative validation. When receiving packets with serial number 5, you may not receive packets with serial number 6, but receive packets later than serial number 7. In this case, ACK can only reply 6, which causes the sender to send data repeatedly. In this case, Sack can be used to solve the problem, which will be discussed below.

The sliding window

The concept of a window is often seen in TCP, as we discussed in the previous section. In TCP, both ends maintain Windows: the sender window and the receiver window respectively.

The sender window contains data that has been sent but not received, and data that can be sent but not sent.

The size of the sender window is determined by the remaining size of the receiver window. The receiver writes the remaining size of the current receiving window into the reply packet. After receiving the reply packet, the sender sets the size of the sending window based on the value and the current network congestion. Therefore, the size of the sending window is constantly changing.

After receiving the reply packet, the sender slides the window

Those of you who have brushed through algorithms should be familiar with these two pictures, because sliding Windows are also a very common problem in algorithms.

Sliding window realizes flow control. The receiver notifies the sender of how much data can be sent through the packet to ensure that the receiver can receive the data in time.

Zero window

During packet sending, a zero window may appear on the peer end. In this case, the sender stops sending data and starts a Persistent timer. The timer periodically sends requests to the peer end to inform the peer end of the window size. If the number of retries exceeds a certain number, the TCP connection may be interrupted.

Congestion control

Congestion processing is a very important function module in TCP. It mainly controls data transmission through some algorithms to prevent network congestion.

Congestion processing is different from traffic control, which works on the receiver to ensure that the receiver can receive data in time. The former works on the network to prevent excessive data congestion and excessive network load.

Congestion processing includes four algorithms: slow start, congestion avoidance, fast retransmission, and fast recovery.

Slow start algorithm

The slowstart algorithm, as its name suggests, slowly and exponentially expands the sending window at the beginning of the transmission, thus avoiding network congestion caused by large amounts of data being transferred at the beginning. For example, in daily downloads, our download speeds are gradually increasing.

The steps of the slow-start algorithm are as follows:

  1. Connection initial set Congestion Window to 1 MSS (maximum amount of data in a segment)
  2. Multiply the window size by two after each RTT
  3. Exponential growth certainly cannot be unlimited, so there is a threshold limit, and congestion avoidance algorithms are activated when the window size is greater than that threshold.

Congestion avoidance algorithm

Congestion avoidance algorithm is simpler and increases the size of each RTT window by one. In this way, network congestion caused by exponential growth can be avoided and the size can be adjusted to the optimal value slowly.

If the protocol considers that the network is congested during transmission, it performs the following steps immediately:

  • Set the threshold to half of the current congestion window
  • Set the congestion window to 1 MSS
  • Enable congestion avoidance algorithm

The fast retransmission

Fast retransmission usually comes with fast recovery. Once the received packets are out of order, the receiver responds only to the sequence number of the last packet (in the case of no Sack). If three duplicate ACKS are received, the peer end does not receive the data sent by the sender end. In this case, fast retransmission is started. The main algorithm is:

TCP Reno

  • Congestion window halved
  • Set the threshold to the current congestion window
  • Enter the fast recovery phase (resending packets needed by the peer end and exiting the phase once a new ACK reply is received)
  • Use congestion avoidance algorithms

TCP New Ren improved fast recovery

The TCP New Reno algorithm improves the defects of the previous TCP Reno algorithm. Previously, fast recovery would exit as soon as a new ACK packet was received.

In TCP New Reno, the TCP sender first jots down the maximum number of the segments of the three repeated ACKS.

If I have a packet with 10 serial numbers from 1 to 10 and packets with serial numbers 3 and 7 are lost, the maximum serial number of the segment is 10. The sender receives only the ACK number 3. In this case, the packet with ACK number 3 is resend. The receiver receives the packet successfully and sends the ACK number 7 reply. In this case, TCP knows that multiple packets are not received from the peer end, and continues to send packets with ACK number 7. The receiver receives packets successfully and sends an ACK number 11 reply. In this case, the sender considers that the segmented receiver has received packets successfully and exits the fast recovery phase.

With TCP out of the way, let’s talk about its sister protocol, UDP. UDP is relatively simple compared with TCP, so it does not involve much knowledge.

UDP

For a message

UDP is a protocol for packets, which can be interpreted as pieces of data. UDP is a porter of packets and does not split or splice packets. That’s what TCP does, after all, to keep the packets in order.

To be specific:

  • At the sending end, the application layer passes the data to the UDP protocol at the transport layer. UDP simply adds a UDP header to the data to identify the UDP protocol, and then passes the data to the network layer
  • At the receiving end, the network layer passes the data to the transport layer, and UDP passes the IP header to the application layer without any concatenation

UDP has some disadvantages because it is very simple to process packets.

Unreliability.

  1. Connectionless, which means that communication does not need to be established and disconnected.
  2. The protocol transmits whatever data it receives and does not back up the data. Does it care whether the other party receives it or not
  3. Without congestion control, data is always sent at a constant speed. Even if the network condition is not good, the sending rate will not be adjusted. The disadvantage of this implementation is that it may lead to packet loss in the case of poor network conditions, but the advantage is also obvious, and it can be very useful in some scenarios with high real-time requirements.

efficient

Because UDP is not as complex as TCP, it is necessary to ensure that data is not lost and arrives in an orderly manner. So UDP header overhead is small, only eight bytes, compared to TCP at least 20 bytes is much less, in the transmission of data packets is very efficient.

The header contains the following data

  • Two hexadecimal port numbers, source port (optional field) and destination port
  • The length of the entire data packet
  • Checksum of the entire data packet (IPv4 optional field). This field is used to find errors in header information and data

With TCP and UDP out of the way, let’s look at what the browser does after the TCP three-way handshake with www.google.com/.

Since the protocol we entered is HTTPS, this protocol can be regarded as HTTP plus TLS. Therefore, after a TCP connection is completed, HTTP transmission is not started immediately, but TLS handshake is started.

HTTPS

The most important part of HTTPS is the TLS protocol, which ensures security.

TLS

Since you need to ensure security, you definitely need to use encryption technology. There are two encryption technologies used in TLS: symmetric encryption and asymmetric encryption.

Symmetric encryption:

Symmetric encryption means that both sides have the same secret key and both sides know how to encrypt and decrypt the ciphertext.

Asymmetric encryption:

There are public key and private key, the public key owner can know, can encrypt the data with public key, but to decrypt the data must use private key decryption, the private key is only known to the party distributing the public key.

TLS handshake process is shown as follows:

  1. The client sends a random value of the required protocol and encryption method
  2. After receiving a random value from the client, the server generates a random value and sends its own certificate based on the protocol and encryption mode required by the client. (Note if the client certificate needs to be authenticated)
  3. After receiving the certificate from the server and verifying whether it is valid, the client generates a random value and encrypts the random value through the public key of the server certificate and sends the random value to the server. If the server needs to verify the client certificate, the server will attach the certificate
  4. The server receives the encrypted random value and decrypts it with the private key to obtain the third random value. At this time, both ends have three random values. The key can be generated by these three random values in accordance with the previously agreed encryption mode, and the subsequent communication can be encrypted and decrypted with this key

It can be seen from the preceding steps that asymmetric encryption is used to communicate with each other during TLS handshake. However, asymmetric encryption has higher performance loss than symmetric encryption, so symmetric encryption is used to communicate with each other during formal data transmission.

PS: The above describes the handshake situation of TLS 1.2 protocol. In 1.3 protocol, only one RTT is required to establish a connection for the first time, and no RTT is required to restore the connection later. Therefore, if you care about network transmission performance, you should choose TLS1.3 protocol.

When TLS completes the handshake, data is actually transferred at the HTTP level.

HTTP

For THE HTTP protocol, front-end engineers mainly know the common status codes and headers, after all, these are the content we often need to contact in our daily coding.

Common status code

2 xx success

  • 200 OK: indicates that the request from the client is processed correctly on the server
  • 204 No content: Indicates that the request is successful, but the response packet does not contain the body of the entity
  • 205 Reset Content: Indicates that the request is successful, but the response packet does not contain the body part of the entity. Different from 204 response, the requestor is required to Reset the Content
  • 206 Partial Content for a range request

3 xx redirection

  • 301 Moved permanently, permanently redirects: indicates that the resource has been assigned a new URL
  • 302 Found, temporary redirection, indicating that the resource was temporarily assigned a new URL
  • 303 See Other: indicates that another URL exists for the resource. Use the GET method to obtain the resource
  • 304 Not Modified: indicates that the server allows access to the resource but the request condition is not met
  • 307 Temporary redirect the client is expected to keep the request method the same and send requests to the new address

4XX Client error

  • 400 Bad Request: Syntax errors exist in the request packet
  • 401 Unauthorized: The request to be sent requires authentication information that is authenticated through HTTP
  • 403 Forbidden: Access to requested resources is denied by the server
  • 404 not found: No requested resource was found on the server

5XX Server error

  • 500 Internal sever error: an error occurred when the server executed the request
  • 501 Not Implemented: The server does Not support a function that is required for the current request
  • 503 Service Unavailable: The server is temporarily overloaded or is being stopped for maintenance and cannot process requests

The HTTP header

General field role
Cache-Control Controls the behavior of caching
Connection The type of connection the browser wants to use preferentially, for examplekeep-alive,close
Date Time of packet Creation
Pragma Packet instructions
Via Proxy server information
Transfer-Encoding Transmission coding mode
Upgrade The client is required to upgrade the protocol
Warning There may be errors in the content
Request field role
Accept The type of media that can be received correctly
Accept-Charset A character set that can be correctly received
Accept-Encoding A list of encoding formats that can be correctly received
Accept-Language List of languages that can be received correctly
Expect Expect the specified behavior of the server
From Email address of the requester
Host Server domain name
If-Match Resource tag comparison at both ends
If-Modified-Since Local resource unmodified returns 304 (comparison time)
If-None-Match Local resource unmodified returns 304 (comparison mark)
User-Agent Client Information
Max-Forwards Limit the number of times that can be forwarded by proxy and gateway
Proxy-Authorization Sends authentication information to the proxy server
Range Request a portion of something
Referer Represents the previous page visited by the browser
TE Transmission coding mode
The response field role
Accept-Ranges Whether certain kinds of scopes are supported
Age The amount of time a resource exists in the proxy cache
ETag Resource identifier
Location The client redirects to a URL
Proxy-Authenticate Sends authentication information to the proxy server
Server Server name
WWW-Authenticate Obtain authentication information required for resources
Entity fields role
Allow The correct way to request a resource
Content-Encoding The encoding format of the content
Content-Language The language in which the content is used
Content-Length The request body length
Content-Location Alternate address to return data
Content-MD5 MD5 check value of content in Base64 encryption format
Content-Range The location range of the content
Content-Type The media type of the content
Expires The expiration time of the content
Last_modified When the content was last modified

The HTTP 2.0

In the last few paragraphs we’ll talk about some new protocols, starting with HTTP 2.0. This protocol significantly improves web performance over HTTP 1.1.

In HTTP 1.1, for performance reasons, we introduced Sprite graphics, inlining small graphs, using domain names, and so on. This is because browsers limit the number of requests for the same domain. When a page has a large number of requests, the Head of line blocking causes the remaining resources to wait for other resources to complete the request when the maximum number of requests is reached.

This problem has been greatly optimized in HTTP 2.0, but remains unsolved. Because HTTP 2.0 is still TCP, TCP’s need to ensure that data is correct also caused the problem of queue blocking, so the problem is not solved. However, this problem was completely solved by the subsequent new protocol, which we will discuss below.

To get a feel for how much faster HTTP 2.0 is than HTTP 1.x, use this link.

In HTTP 1.x, you’ll find requests like this because the queue header is blocked

In HTTP 2.0, because of the introduction of multiplexing, you’ll find requests that look like this:

multiplexing

In HTTP 2.0, there are two very important concepts: frame and stream.

A frame represents the smallest unit of data, and each frame identifies which stream it belongs to. A stream is a data stream composed of multiple frames.

Multiplexing means that multiple streams can exist in a TCP connection. In other words, multiple requests can be sent, and the peer end can know which request belongs to by the identifier in the frame. The transmission performance can be greatly improved through this technology.

Binary transmission

This is the core of all performance enhancements in HTTP 2.0. In previous versions of HTTP, we transferred data as text. A new encoding mechanism was introduced in HTTP 2.0, where all transmitted data is split and encoded into binary frames in a binary format.

There are many types of binary frames, such as HEADERS frames and DATA frames.

The Header compression

In HTTP 1.x, we transmit the header as text, and in cases where the header carries cookies, we may need to repeat the transfer of hundreds to thousands of bytes at a time.

In HTTP 2.0, the transmitted headers are encoded using the HPACK compression format, reducing their size. An index table is maintained at both ends to record the headers that have been recorded. The key names of recorded headers can be transmitted later in the transmission process. After receiving data, the peer end can find the corresponding value by the key names.

The server Push

You don’t need to learn this, because you don’t use it enough, Chrome may remove this feature. See Chrome to Remove HTTP/2 Push for details.

QUIC

This is a Google-made udP-based transport-layer protocol with the ambitious goal of replacing TCP.

  • This protocol supports multiplexing, although HTTP 2.0 also supports multiplexing, but the lower layer is still TCP, because TCP retransmission mechanism, as long as a packet is lost to judge the loss of packets and retransmission, resulting in the problem of queue head blocking, but UDP does not have this mechanism
  • It implements its own encryption protocol, and 0-RTT can be achieved through a TFO mechanism similar to TCP, that is, QUIC can establish a secure connection and transmit data in 0-RTT
  • Support for retransmission and error correction mechanisms (forward recovery). If only one packet is lost, retransmission is not required and the error correction mechanism is used to recover the lost packet
  • Error correction mechanism: The server calculates the xOR value of outgoing data by xOR method and sends a packet separately. If a packet is found missing, the server calculates the missing packet by other packets and xOR value packet
  • Use the retransmission mechanism in the case of two or more missing packets, because you can’t calculate it

The last

There’s a lot more to network protocols, and HERE’s what front-end engineers have to learn. If you’re interested in a particular protocol you can study it for yourself.

A few examples:

  1. There’s a lot more to HTTP 2.0, such as the different types of binary frames and what they do.
  2. HTTP 3.0 is a newer protocol with many performance improvements compared to 2.0.
  3. QUIC is a protocol with strong performance and TCP functions. Some CDN vendors already support enabling this function in the background.

Throw some study leads to the reader. If you have any doubts about the above content, you can exchange and learn together.

You can also read on the author’s website, better experience!