Follow the public number “kite holder”, get a lot of teaching videos and enter the professional communication group.
What happens when the browser enters a URL request?
2.DNS domain name resolution
(1) Search the DNS cache of the browser
(2) If the browser cache does not contain the IP address mapping, the operating system will first check whether the local hosts file has the IP address mapping. If yes, the operating system will call the IP address mapping to complete the domain name resolution.
(3) If the hosts does not contain the mapping of the domain name, check whether the local DNS resolver cache has the mapping. If yes, return the mapping directly to complete the domain name resolution.
(4) If there is no corresponding url mapping relationship between hosts and the local DNS resolver cache, the local DNS server is found. If the domain name to be queried is contained in the local configuration area resources, the resolution result is returned to the client to complete the domain name resolution. The resolution is authoritative.
(5) If the domain name to be queried is not resolved by the local DNS server, but the server has cached the url mapping, the IP address mapping is invoked to complete the domain name resolution. The resolution is not authoritative.
Failure, (6) if the above methods are carried out by local DNS server iterative queries, first request to the root domain DNS server, check the secondary domain, three domains, until the query to the address or the name of parsing, local DNS server after receiving the response, stored in the cache first, and then parse the results back to the client.
3. Establish a TCP connection (three-way handshake)
The TCP protocol uses the three-way handshake policy. The sender sends a packet with the SYN (synchronize) flag to the receiver. After receiving the packet, the receiver sends back a packet with the SYN/ACK flag (acknowledegment) to convey the acknowledgement. Finally, the sender sends back a packet with an ACK flag to indicate the end of the handshake.
A complete HTTP request consists of a request starting line, a request header, and a request body.
Request method:
GET: Obtains resources
POST: indicates the transfer entity body
HEAD: indicates the packet header
PUT: transfers files
DELETE: deletes a file
OPTIONS: Asks about supported methods
TRACE: TRACE path
5. The server processes the Request and the browser receives the HTTP Response. After receiving the HTTP Request sent by the browser, the server encapsulates the received HTTP packet into AN HTTP Request object and processes it through different Web servers. It consists of status code, response header and response packet. Status code: 1xx: Indicates that the request has been received and needs to be processed. 2xx: success status code
200– OK/ the request has been processed normally 204– the request was processed successfully, but no resources were returned 206– indicates that the client made the scope request and the server successfully executed this part of the GET request
3xx: indicates the redirection status code
301– / request permanent redirection The requested resource has been permanently moved to a new location, and any future reference to this resource should use one of the urIs returned by this response. If possible, clients with link editing capabilities should automatically change the requested address to the address returned from the server. 302– / Request temporary redirection Since such redirection is temporary, the client should continue to send future requests to the original address. The response is cacheable only if it is specified in cache-control or Expires. 304– / If the client sends a conditional request (if-match, if-modified-since, if-none-match, if-range, if-unmodified-since any header), The server allows the request to access the resource, but the condition is not met 307- temporary redirection, same as 302, but 307 does not change from POST to GET according to browser standards
4xx: indicates the client error status code
400– / client request has syntax error 401– / current request requires user authentication. 403– / The server understood the request but refused to perform it. Unlike the 401 response, authentication does not provide any help 404– / The request failed and the desired resource was not found on the server. 405– / The request method specified in the request line cannot be used to request the corresponding resource.
5xx: indicates the server error status code
500– / The server encountered an unexpected situation that caused it to be unable to complete processing the request.
501– / The server does not support a feature required by the current request. When the server cannot recognize the method of the request and cannot support its request for any resource.
503– / The server is currently unable to process requests due to temporary server maintenance or overload.
505– / The server does not support, or refuses to support, the HTTP version used in the request.
6. The browser parses the rendered page
(1) Building the Document Object Model (DOM)
(2) Build CSS Object Model (CSSOM)
(3) Build a Render Tree
(4) Layout
(5) Drawing
7. End of connection (4 waves)
Close the connection with four waves
The first wave is when the browser finishes sending data and sends a FIN request to disconnect.
The second wave is the server sending an ACK of approval.
The third wave is a FIN request sent by the server to disconnect the connection. (Considering that the server may still have data to send)
The fourth wave is when the browser needs to return an ACK indicating consent.
2. HTTP cache?
HTTP caches have a variety of rules that classify them into forced caches and comparison caches based on whether or not a request needs to be resent to the server.
(1) Mandatory cache: determine the HTTP header fields: cache-control and Expires
Max-age in cache-control stores a relative time. For example, cache-control: max-age = 484200, the Cache is valid for 484200 seconds after the browser receives the file. If both cache-control and Expires are present, the browser always prefers cache-control.
Cache-control is the most important rule.
Common values include private, public, no-cache, max-age, and no-store. The default value is private.
Private: The client can cache
Public: Both the client and proxy servers can be cached.
Max-age = XXX: The cached content will be invalid after XXX seconds
No-cache: Need to use contrast cache to validate cached data (described below)
No-store: Nothing is cached, force caching, comparison caching is not triggered (for front-end development)
(2) Compare cache: Judge by last-Modified and Etag fields of HTTP
Last-modified is the field returned by the server when the resource was first requested, indicating the last updated time. The next time the browser requests a resource, it sends the IF-modified-since field. The server compares the local last-modified time with the if-modified-since time. If there is a difference, the server considers the cache expired and returns a new resource to the browser. If the time is consistent, send a 304 status code and let the browser continue to use the cache.
Etag: The entity identifier (hash string) of the resource. When the resource content is updated, the Etag changes. The server determines whether the Etag has changed and returns a new resource if it has, or 304 if not.
3. DNS server
1. Why do YOU need DNS to resolve a domain name to an IP address
Network communication is mostly based on TCP/IP, and TCP/IP is based on IP address, so computers can only recognize IP addresses but not domain names when communicating on the network.
2. The DNS
DNS is the domain name system, which provides services for translating host names and domain names into IP addresses.
3. Suppose that some application (such as a Web browser or mail reader) running on a user’s host needs to translate the host name into an IP address. These applications invoke the client side of DNS and specify the host name that needs to be transformed. After receiving the packet, the DNS client of the user host sends a DNS query packet to the network. The UDP datagram used for all DNS request and reply packets is sent through port 53. After a delay of several ms to several s, the DNS client on the user host receives a DNS reply packet that provides the desired mapping.
4.DNS works in a distributed cluster rather than a single point of centralized design, because centralized design may have single points of failure, communication capacity, long distance time delay, and high maintenance overhead.
5. The following figure shows the DNS query process
When a host makes a DNS request, the request is sent to the local DNS server, which acts as a proxy and forwards the request to the DNS server hierarchy.
DNS provides two kinds of query procedures:
Recursive query: In this mode, the DNS server receives a client request and must reply the client with an exact query result. If the DNS server does not store a DNS value, the DNS server will ask other servers and return a query result to the client.
Iterative query: The DNS server provides other DNS servers that can interpret the query request to the client. When the client sends a query, the DNS does not directly reply to the query result. Instead, the DNS tells the client the address of another DNS server.
Four, TCP and UDP differences, advantages and disadvantages
4.1 Summary of TCP and UDP
(1) TCP connection-oriented; UDP is connectionless, that is, no connection needs to be established before sending data. (2) TCP provides reliable data transmission. By using flow control, serial number, acknowledgement and timer, TCP ensures that the data is delivered correctly and sequentially from the sending process to the receiving process; UDP tries its best to deliver, that is, does not guarantee reliable delivery; (3) UDP has better real-time performance and higher work efficiency than TCP, which is suitable for high-speed transmission and high real-time communication or broadcast communication; (4) Each TCP connection can only be point-to-point. UDP supports one-to-one, one-to-many, many-to-one and many-to-many interactive communication. (5) TCP requires more system resources, while UDP requires less system resources.
4.2 UDP Application Scenarios
(1) Datagram oriented mode (2) Network data is mostly short messages (3) a large number of clients (4) no special requirements for data security (5) The network burden is heavy, but the response speed is high
4.3 How does TCP Provide Reliable Data Transmission
TCP ensures the correct and sequential delivery of data from the sending process to the receiving process by using flow control, numbering, acknowledgements, and timers.
The TCP sender has three events related to the send and retransmission
Receives data from upper-layer applications
TCP receives the data from the application, encapsulates it in a segment (containing the stream number of the first data byte), and delivers it to the IP.
Timer timeout
After the timeout, TCP retransmits the timeout packet and restarts the timer.
Receives an ACK
After receiving an ACK, the device compares the acknowledgement number in the acknowledgement packet with SendBase (the number of the first unacknowledged byte) of the sender. TCP takes cumulative acknowledgment, so all bytes before the acknowledgment number are received by the receiver. When the acknowledgement number is greater than SendBase, the ACK is confirming one or more previously unacknowledged packet segments. In this case, the sender updates the value of SendBase. If there are currently unacknowledged packet segments, the TCP restarts the timer
Six, TCP protocol in the working process of several simple cases
6.1 Retransmission due to Confirmation loss
6.2 ACK Delay for Continuously Sent Segments
6.3 Cumulative Acknowledgement Prevents retransmission of previous segments
7. Quick retransmission
One of the problems with timeout retransmission is that the timeout period can be long. When a segment is lost, the recovery packet is retransmitted through timeout, which increases the end-to-end delay. Luckily, we can retransmit the lost segment by detecting the received redundant ACK.
The reason why you can be sure that the segment is lost is:
(1) After receiving the lost packet segment (that is, the out-of-order packet segment), the receiver caches the out-of-order packet segment and sends the maximum number of the most recently received unout-of-order packet segment to the sender.
(2) If the receiver continuously receives multiple out-of-order packets, the sender receives multiple ACK’s for the same packet segment. In this way, the sender knows that the last packet of the packet segment represented by the ACK is lost. Therefore, the sender retransmits the lost packet.
When the sender receives three redundant ACK packets, it indicates that the packet segment after the packet segment that has been acknowledged for three times is lost. TCP performs fast retransmit to retransmit the lost packet segment before the timeout of the packet segment timer.
8. Whether to go back N steps or choose retransmission in TCP
According to the previous DESCRIPTION of TCP, the TCP acknowledgement is cumulative and does not acknowledge out-of-order packets. This makes TCP look like a GBN protocol, but unlike GBN, TCP will cache out-of-order packets. Therefore, one modification proposed by TCP is to select slective Acknowledgment [RFC 2018], which allows THE TCP receiver to selectively acknowledge out-of-order packet segments instead of accumulating acknowledgment of the last properly received ordered packet segment. When used in combination with the selective retransmission mechanism (that is, hop retransmission of segments that have been selected by the recipient to acknowledge), TCP looks like our usual SR protocol. Therefore, the error recovery mechanism of TCP is a mixture of GBN and SR protocols.
Flow control and congestion control in TCP
They are both deterrents to the sender
9.1 Flow Control (to avoid receiver cache overflow)
Simply put, flow control is provided to avoid the receiver cache overflow problem. After receiving the data, the receiver puts it into the receive cache for the upper-layer application to read. However, the upper-layer application may be busy with other transactions or read data slowly, and the sender may send too much data at too fast a rate, causing the receiver cache to overflow. Flow control is also a rate matching service. 2. How does TCP provide the traffic control service? For overall purposes, we assume that the TCP receiver discards out-of-order packets.
TCP lets sender A maintain A variable called receive Window to provide flow control. After A TCP connection is established between host A and host B, host B allocates A receive buffer for the connection, which is represented by the RcvBuffer
Define the following variables
LastByteRead: Number of the last byte of the data stream that host B’s application process pulled out of the cache. LastByteRevd: number of the last byte of the data stream that host B cached
Lastbyterevd-lastbyteread <= RevBuffer receive window RWND set according to the available cache space: RWND = RevBuffer – [lastByterevD-lastByterEAD]
Host B has notified host A of how much space it currently has available by placing the current RWND in the field of the receive window of the segment it sends to host A. Host A always tracks two LastByteSend and LastByteAcked. [lastBytesend-lastByteAcked] is the amount of data sent by Host A but not confirmed. If this value is smaller than the RWND of host B, host B’s cache will not overflow.
9.2 Congestion Control
TCP senders may also be curbed due to IP network congestion; this form of control is known as congestion control.
General principle: Congestion control occurs because resources (bandwidth, switching node cache, processor) are required > available resources.
1. Congestion sign
The retransmission timer timed out or three double acknowledgements were received.
2. Avoid slow start and race
(1) Slow start does not mean that CWND (Congestion window) grows slowly (exponentially), but means that TCP starts sending with CWND =1.
(2) Idea: Do not send a large amount of data at the beginning, first detect the network congestion degree, that is to say, gradually increase the size of the congestion window from small to large.
(3) To prevent network congestion caused by CWND growth, set a slow start threshold
When CNWD < SSthRESH, use the slow start algorithm
When CNWD = SSthRESH, either the slow start algorithm or the congestion avoidance algorithm can be used
When CNWD > SSthRESH, use the congestion avoidance algorithm
3. Congestion avoidance
(1) Congestion avoidance is not completely able to avoid congestion. It means that in the phase of congestion avoidance, the congestion window is controlled to grow according to the linear law, so that the network is less prone to congestion.
(2) Idea: Let the congestion window CWND increase slowly, that is, every round trip time RTT will increase the sender’s congestion control window by one.
Either in the slow start phase or in the congestion avoidance phase, as long as the sender determines that the network is congested, the slow start threshold is set to half the size of the send window when the congestion is present. Then set the congestion window to 1 and perform the slow start algorithm.
(1) Fast retransmission requires the receiver to send a double acknowledgement immediately after receiving a misordered segment (so that the sender can know in advance that a segment has not reached the recipient) rather than waiting for a piggy-back acknowledgement when sending the data. According to the fast retransmission algorithm, as long as the sender receives three consecutive double acknowledgements, it should immediately retransmit the packet segments that have not been received by the sender without waiting for the expiration of the retransmission timer.
(2) Because there is no need to wait for the set retransmission timer to expire, unacknowledged packet segments can be retransmitted as early as possible, which can improve the throughput of the whole network.
Fast recovery (used in conjunction with fast retransmission)
(1) When the fast recovery algorithm is used, the slow start algorithm is used only when the TCP connection is established and the network times out.
(2) When the sender receives three consecutive repeated acknowledgments, the “multiplication reduction” algorithm is executed to halve the SSthresh threshold. But the slow start algorithm is not executed next.
(3) The sender now thinks that the network may not be congested, taking into account that it will not receive several repeated acknowledgments if the network is congested. So instead of executing the slow start algorithm at this time, CWND is set to the size of SSthRESH and then the congestion avoidance algorithm is executed.
5. Congestion window
The sender is a dynamically changing window called a congestion window. The size of the congestion window depends on the degree of network congestion. The sender to send window = congestion window, but send window is not always equal to the congestion window, in the network situation, the congestion window increasing, nature also increases as the sender’s window, but the person accept ability is limited, in the sender’s window reaches a certain size would not be in change.
9.3 Differences between Congestion Control and Flow Control
Congestion control is a global process that prevents excessive data from being injected into the network and prevents routers or links on the network from being overloaded. Flow control is the control of point-to-point traffic. It is an end-to-end problem. The main problem is to suppress the rate at which the sender sends data so that the receiver can receive the data in time.
TCP three handshakes and four waves
Acknowledge the ACK. The acknowledgement number field is valid only if ACK=1. TCP specifies that after a connection is established, all packets must be transmitted with ACK set to 1. Synchronization SYN, used to synchronize serial numbers when a connection is established. If SYN=1 and ACK=0, it indicates a connection request packet. If the connection is agreed, then SYN=1 and ACK=1 should be set in the response packet. Terminate the FIN to release the connection. When FIN=1, it indicates that the data of the sender of the packet has been sent and requires release.
10.1 Three handshakes
Both the client and server are in a CLOSED state to begin with. Actively open the connection for the client, passively open the connection is the server.
2. The TCP client process first creates the transmission control block TCB, and then sends a connection request packet to the server. At this time, the same part in the packet header is SYN=1, and the initial sequence number is SEq =x. According to TCP, the SYN packet segment (SYN=1) cannot carry data, but consumes a sequence number.
3. After receiving the request packet, the TCP server sends an acknowledgement packet if it agrees to the connection. In the acknowledgement packet, ACK=1, SYN=1, ACK= x+1, and seq=y should be initialized. At this time, the TCP server process enters syn-RCVD (synchronously received) state. This packet also does not carry data, but it also consumes a serial number.
4. After receiving the acknowledgement, the TCP client process also sends the acknowledgement to the server. When ACK=1, ACK= y+1, and seq=x+1, the TCP connection is ESTABLISHED and the client enters the ESTABLISHED state. According to TCP, AN ACK packet segment can carry data, but no sequence number is consumed if it does not carry data.
5. After receiving the confirmation from the client, the server enters the ESTABLISHED state. After that, the communication between the server and the client starts.
Extension: Why does a TCP client send a final acknowledgement?
In short, the main purpose is to prevent the connection request packet that has failed to be sent to the server suddenly, resulting in error.
If you are using two handshake connection is established, suppose there is such a scenario, the client sends the first request connection and not lost, just because in the network node in the retention time is too long, as a result of the TCP client has not received the confirmation message, thought the server did not receive, at this time to send this message to the server, After that, the client and server complete the connection through two handshakes, transfer data, and then close the connection. At this point, the previous delayed request connection, the network is normal and reaches the server. This packet should be invalid. However, the two-time handshake mechanism will make the client and the server establish a connection again, which will lead to unnecessary errors and waste of resources.
If the three-way handshake is used, even if the invalid packet is sent, the server receives the invalid packet and replies with an acknowledgement packet, but the client does not send acknowledgement again. Because the server does not receive an acknowledgement, it knows that the client did not request a connection.
10.2 Four waves
2. After receiving the connection release packet, the server sends an ACKNOWLEDGEMENT packet with ACK=1, ACK= U +1 and its serial number seq= V. Then the server enters the close-wait state. The TCP server notifies the high-level application process, and the client is released to the server. At this time, the client is in the semi-closed state, that is, the client has no data to send. However, if the server sends data, the client still accepts it. This state will continue for a period of time, that is, the entire duration of the closewait state.
3. After receiving the acknowledgement request from the server, the client enters the FIN-wait-2 state, waiting for the server to send the connection release packet (before receiving the last data sent by the server).
4. After the server sends the LAST data, it sends the connection release packet to the client, FIN=1, ACK = U +1. Since the server is in the semi-closed state, it is likely to send some more data, assuming that the serial number at this time is SEq = W.
5. After receiving the connection release packet from the server, the client must send ACK=1, ACK= w+1, and its serial number is SEq = U +1. In this case, the client enters the time-wait state. Notice That the TCP connection is not released, and the client enters the CLOSED state only after 2∗∗MSL (maximum packet segment lifetime) expires and the corresponding TCB is revoked.
6. Once the server receives the acknowledgement from the client, it enters the CLOSED state immediately. Again, canceling the TCB terminates the TCP connection. As you can see, the server ends the TCP connection a little earlier than the client.
Extensions: Why does the client end up waiting for 2MSL?
Maximum Segment Lifetime (MSL) : TCP allows different implementations to set different MSL values.
First, ensure that the client sends the final ACK packet to reach the server, because the ACK packet may be lost, standing in the server’s perspective, I have already sent the FIN + ACK message request disconnect, the client also did not give me response, should be I send the request of the disconnect message it did not receive, then the server will send a again, The client will receive the retransmitted packet within the 2MSL period, then send a response packet and restart the 2MSL timer.
Second, to prevent “invalid connection request segments” such as those mentioned in the “three-way handshake” from appearing in this connection. After the client sends the last acknowledgement packet, all the packet segments generated during the duration of the connection can disappear from the network in the 2MSL time. In this way, the request packet of the old connection does not appear in the new connection.
10.3 Why Is a Three-way Handshake used to Establish a Connection and a Four-way Wave used to Close a Connection?
During connection establishment, the server is in LISTEN state. After receiving a SYN packet, the server sends an ACK packet and a SYN packet to the client. And close connection, the server receives the other side of the FIN message, just said to each other can no longer send data but also receives the data, and the oneself also is not necessarily all data are sent to each other, so their can immediately shut down, also can send some data to each other, then send the FIN message now agreed to close the connection to the other side, therefore, Your OWN ACK and FIN are usually sent separately, resulting in one more time.
10.4 What If the Client Fails after the Connection is Established?
TCP also has a keepalive timer, so obviously, if a client fails, the server can’t wait and waste resources. The server resets the timer every time it receives a request from the client. The time is usually set to 2 hours. If no data is received from the client within 2 hours, the server sends a probe segment and then sends it every 75 minutes. If there is no response after 10 consecutive probes, the server considers the client to be faulty and closes the connection.
11. HTTP Protocol (Request and Response Packets)
11.1 Request Packet
HTTP request packets include request lines, request headers, and request data (entities).
1. Request line: Method field, URI field, and protocol version
Method field: GET (request to obtain content), POST (submit form), HEAD (request resource response message header), PUT (transfer file), DELETE (request to DELETE the resource pointed to by the URI), OPTIONS (query methods supported for the resource specified by the request URI), TRACE (TRACE the request path), and CONN ECT (Requires a tunneling protocol to connect to a proxy)
2. Request header
Common headers are: The Connection header (Connection management), the Host header (which specifies the Host of the requested resource), the Range header (the Range of bytes of the requesting entity), the User-Agent header (which contains information about the User making the request), the Accept header (preferred media type), and the Accept-Language(preferred from But language)
application/json
application/x-www-form-urlencoded
multipart/form-data
text/xml
The HTML request generated by HTML form tag submission will generate Application/X-wwW-form-urlencoded data format by default, and multipart/form-data will be used when a file is uploaded.
11.2 Response Packet
The HTTP response packet is divided into three parts: the status line, the header line, and the entity.
1. Status line: Version, status code, and cause statement
Status code:
(1) 1xx: This type of status code indicates that the request has been accepted and needs to continue processing. This type of response is temporary, contains only the status line and some optional response header information, and ends with a blank line;
(2) 2xx: status code of this type, which means that the request has been successfully received, understood and accepted by the server;
(3) 3xx: This type of status code indicates that the client needs to take further action to complete the request. Typically, these status codes are used for redirection, and subsequent request addresses (redirection targets) are specified in the Location field of this response.
(4) 4xx: This kind of status code represents the client class error;
(5) 5xx: Server class error.
200– OK/ The request has been processed normally
204– The request was processed successfully, but no resources returned
206- Indicates that the client made a scope request and the server successfully executed this part of the GET request
301– / request permanent redirection The requested resource has been permanently moved to a new location, and any future reference to this resource should use one of the urIs returned by this response. If possible, clients with link editing capabilities should automatically change the requested address to the address returned from the server.
302– / Request temporary redirection Since such redirection is temporary, the client should continue to send future requests to the original address. The response is cacheable only if it is specified in cache-control or Expires.
303– Indicates that the requested resource has another URI and should be directed to the requested resource using the GET method
304– / If the client sends a conditional request (if-match, if-modified-since, if-none-match, if-range, if-unmodified-since any header), A condition in which a server allows a request to access a resource, but the condition is not met
307- temporary redirect, same as 302, but 307 does not change from POST to GET according to browser standards
400– / The client request has syntax errors
401– / The current request requires user authentication.
403– / The server understood the request but refused to perform it. Unlike the 401 response, authentication does not help
404– / The request failed, the requested resource was not found on the server.
405– / The request method specified in the request line cannot be used to request the corresponding resource.
500– / The server encountered an unexpected situation that caused it to be unable to complete processing the request.
501– / The server does not support a feature required by the current request. When the server cannot recognize the method of the request and cannot support its request for any resource.
503– / The server is currently unable to process requests due to temporary server maintenance or overload.
505– / The server does not support, or refuses to support, the HTTP version used in the request.
2. Response header
Date, Via, last-Modified, Etag, Connection, accept-ranges, and Content- Type (resource Type)
Cookie and Session
12.1 Cookie size 4KB
The working mechanism of Cookie is user identification and state management.
This header field is used to tell the client all kinds of information about the Cookie
This header field is used to tell the server that when the client wants HTTP state management support, it will include the Cookie received from the server in the request.
3.HTTP is a stateless protocol. That is, the HTTP protocol itself does not save the communication state between the request and response. But in order to achieve the desired state-holding capability, Cookie technology was introduced.
The Cookie notifies the client to save the Cookie according to the header field information called set-cookie in the response packet sent from the server. When the client sends a request to the server next time, the client automatically adds the Cookie value to the request packet and sends the request packet.
12.2 the Session
Sessions are another mechanism for recording a client’s state, except that cookies are stored in the client’s browser and sessions are stored on the server. When the client browser accesses the server, the server records the client information in some form on the server. So this is Session. The client browser only needs to look up the state of the client from the Session when it accesses the Session again. 1. How does the backend store sessions? An address space is created in the server memory to store the private data of each client. Each client can obtain session data based on the session ID maintained in the cookie.
12.3 Differences between Cookies and Sessions
1) Cookie and Session are both Session technologies, with Cookie running on the client and Session running on the server. 2) There is a 4K limit on the size of cookies and a limit on the number of cookies stored by the browser. There is no limit on the size of Session, which is related to the server’s memory size. 3), cookies have security risks, through the interception or local files to find your cookies can be attacked. 4) Sessions are stored on the server for a period of time before they disappear. Too many sessions will increase the pressure on the server. Sessions are used to store important information, and cookies are used to store unimportant user information
13. Login verification
Since HTTP is stateless, a request ends, the connection is disconnected, and the next time the server receives a request, it does not know which user sent the request. Therefore, state management is needed so that the server can know exactly which user initiated the HTTP request and determine whether the user has permission to continue the request. This process is often referred to as session management.
13.1 Login in the Same Domain
If the front-end and back-end apis are deployed in the same domain without cross-domain deployment, the login is relatively simple
1. Log in based on Session
The server uses Session technology and the browser uses Cookie technology.
In fact, the above can be explained as:
When the user logs in, he/she brings his/her own user name and password, stores the login credentials information of the user in the Session, generates cookies, and returns in the response through set-cookie; On the second login, add a Cookie to the request and send it. The server checks the Cookie and then responds.
2. Log in based on the Token
(2). The front-end obtains the Token, stores it in cookie or localStorage, and passes the Token to the server through URL parameter or HTTP Header in the following request
(3). The server obtains the token value and determines whether the current token is valid by searching the database
13.2 Cross-Domain Login
Due to the same origin policy of the browser, if the protocol, domain name, and port number of the URL that sends the request are different from the current page, the request is considered as cross-domain 1. Address the same Origin policy For both Session based and Token based logins by setting Header on the server and access-control-allow-origin if you want to send cookies, Access-control-allow-origin cannot set asterisks, and must specify an explicit domain name that is consistent with the requested web page. By default, CORS does not send cookies and HTTP authentication information. On the one hand, the server sets access-control-allow-credentials: On the other hand, the developer must add the withCredentials attribute to the Ajax request.
Fourteen, HTTP1.0, HTTP1.1, HTTP2.0
14.1 Basic HTTP optimization
HTTP network requests are affected by two main factors: bandwidth and latency 1. Bandwidth has been greatly improved 2. Delay (1) Browser blocking: For the same domain name, the browser can only have 6 connections (different browsers) at the same time. If the maximum number of connections exceeds the limit, the subsequent requests will be blocked. (2) DNS query: The browser needs to know the IP address of the target server in order to establish a connection, often using DNS cached results to reduce this time. (3) Connection establishment: HTTP is based on the TCP protocol, the fastest browser can pick up the HTTP request packet on the third handshake, to achieve the actual connection establishment, but these connections can not be multiplexed, each request will go through three handshakes and slow start. The impact of the three-way handshake is obvious in the scenario with high delay, while the impact of the slow start is significant in the scenario with large file requests.
14.2 Some differences between HTTP1.0 and HTTP1.1
HTTP1.0 uses if-modified-since,Expires in headers as the standard for caching. HTTP1.1 introduces more cache control strategies such as Etag. If-unmodified-since, if-match, if-none-match, and many more alternative cache headers to control cache policies. In HTTP1.0, there are some waste of bandwidth phenomenon, such as the client only needs a part of an object, but the server will send the whole object, and do not support the resumable function, HTTP1.1 in the request header introduced the range header field, it allows only a part of the resource request, HTTP1.0, there are some waste of bandwidth phenomenon, such as the client only needs a part of an object, but the server will send the whole object, and do not support the resumable function, HTTP1.1 in the request header introduced the range header field, it allows only a part of the resource request. The return code is 206 (Partial Content), which gives the developer the freedom to make the most of the bandwidth and connection. 3. Error notification management in HTTP1.1 added 24 error status response codes, such as 409 (Conflict) indicates that the requested resource and the current state of the resource Conflict; 410 (Gone) Indicates that a resource on the server has been permanently deleted. HTTP1.0 assumes that each server is bound to a unique IP address, so the URL in the request message does not pass the hostname. However, with the development of virtual hosting technology, there can be multiple homed Web Servers on a physical server, and they share an IP address. HTTP1.1 should support the Host header field in both Request and response messages, and a Request message without the Host header field will report an error (400 Bad Request). HTTP1.1 supports PersistentConnection and pipelinedprocessing. Multiple HTTP requests and responses can be transmitted over a single TCP Connection, reducing the consumption and latency of establishing and closing connections. Connection is enabled by default in HTTP1.1. Keep-alive, to some extent, makes up for the drawback of HTTP1.0 having to create a connection on every request.
14.3 Extensions: Several features of HTTP1.1
1. Persistent Connections Each TCP connection starts with three handshakes and requires a full round trip between the client and the server. With a persistent connection open, there is no need to shake hands every time.
Persistent Connection Connection: keep-alive
Persistent HTTP multiple requests must strictly comply with a first-in-first-out (FIFO) queue order: send the request, wait for the response to complete, and then send the next request in the client queue.
The HTTP pipeline allows us to migrate the FIFO queue from the client (request queue) to the server (response queue).
However, HTTP 1.x does not allow multiple response data on a connection to be interleaved (multiplexing), so one response must be fully returned before the next one is transmitted.
14.4 HTTP Bottlenecks
Only one request can be sent on a connection; Requests can only start from the client. The client cannot receive instructions other than responses. The request/response header is sent uncompressed. The more information in the header, the greater the delay; Send lengthy headers. Sending the same header to each other every time causes more waste; You can choose any data compression format. Non-forced compression send;
14.5 HTTP2.0
HTTP2.0 is the next generation Internet communication protocol formed on the basis of SPDY. The purpose of HTTP/2 is to reduce latency by enabling multiplexing of requests and responses, reduce protocol overhead by compressing HTTP header fields, and increase request priority and server push support.
New features of HTTP2.0 relative to Http1.x
1. Binary frame layer
The binary framing layer is at the heart of HTTP2.0’s performance enhancements
Whereas HTTP 1.x communicates as plain text at the application layer, HTTP 2.0 splits all the transmitted information into smaller messages and frames and encodes them in a binary format. Thus, both the client and the server need to introduce new mechanisms for binary encoding and decoding
(1) frame
The minimum unit of HTTP2.0 communication, including the frame header, stream identifier, priority value, and frame load.
DATA: used to transmit the HTTP message body.
HEADERS: Transmits the header field.
SETTINGS: Used to specify the configuration data of the client and server. For example, set the initial two-way flow control window size;
WINDOW_UPDATE: Used to adjust traffic for individual streams or connections
PRIORITY: Specifies or respecifies the PRIORITY of the referenced resource.
RST_STREAM: Used to notify an abnormal termination of a stream.
PUSH_ PROMISE: the server pushes a license.
PING: calculates the round-trip time and performs the active check.
GOAWAY: Notifies the peer to stop creating streams in the current connection.
(2) Messages
A message is a logical HTTP message (request/response). A series of data frames makes up a complete message.
(3) the stream
A stream is a virtual channel in a connection that can host two-way message transmission. Each stream has a unique integer identifier. To prevent stream ID conflict between the two ends, the stream initiated by the client has an odd ID and the stream initiated by the server has an even ID.
All HTTP2.0 communication is done over a TCP connection that can host any number of two-way data streams. Accordingly, each data stream is sent as a message, which consists of one or more frames that can be sent out of order and then reassembled according to the stream identifier at the head of each frame.
2. Multiplexing Shared Connections THE HTTP message is broken up into individual frames without breaking the semantics of the message itself, interleaving them, and finally recombining them at the other end based on the stream ID and first part.
HTTP2.0 successfully solves the http1.x queue-first blocking problem (TCP layer blocking is still unresolved), and reduces the number of TCP connections to greatly improve server performance.
- Requests can be sent in parallel and interleaved, and each request does not affect the other; * Responses can be sent in parallel and interleaved, and each response does not interfere with the other; * Send multiple requests and responses in parallel using only one connection; * Reduce page loading time by eliminating unnecessary delays; * Don’t have to do much more work to bypass HTTP 1.x restrictions;
3. The request priority stream can have a 31bit priority: 0: indicates the highest priority 2 ^ 31 -1 indicates the lowest priority.
The server returns results based on priority, which helps to efficiently utilize underlying connections and improve user experience.
4. Server push
HTTP2.0 adds the server push function, according to the client request, the server can return multiple responses in advance, push additional resources to the client.
All pushed resources comply with the same origin policy. In other words, the server cannot push third-party resources to the client, but only after both parties confirm.
The PUSH_PROMISE frame is a signal that the server is intentionally pushing a resource to the client.
If the client does not need a server Push, disable this function by setting the server stream value to 0 in the SETTINGS frame
The PUSH_PROMISE frame contains only the header of the pre-pushed resource. If the client has no objection to the PUSH_PROMISE frame, the server sends a DATA frame after the PUSH_PROMISE frame to push the resource. If the client has already cached the resource and does not need to push it, you can reject the PUSH_PROMISE frame.
PUSH_PROMISE must follow the request-response principle, pushing resources only in response to requests.
A few restrictions:
The server must follow a request-response cycle, pushing resources only in response to requests
The PUSH_PROMISE frame must be sent before the response is returned to avoid race conditions on the client.
Each http1.x communication (request/response) carries header information to describe resource attributes. HTTP2.0 uses a “header table” between client and server to track and store previously sent key-value pairs. The header table is always present during the join process, and new key-value pairs are updated to the end of the table. Therefore, there is no need to carry the header in every communication. HTTP2.0 uses header compression technology to make headers more compact and faster to transmit. HTTP 2.0 focuses on header compression, while gZIP and other commonly used compression is the body of the packet. Two, the complete HTTP2.0 communication process in the two ends of the use of HTTP2.0 communication, there must be a process of protocol negotiation. 1. Alpn-based negotiation During HTTPS negotiation, ALPN (Application Layer protocol negotiation) is used. Reducing network latency is a key condition of HTTP 2.0, so ALPN negotiation must be used when establishing HTTPS connections. Browsers that support HTTP 2.0 can automatically complete protocol negotiation with the server at the TLS session layer to determine whether to use HTTP 2.0 communication. The principle is that TLS 1.2 introduces the extension field to allow the extension of the Protocol. The Application Layer Protocol Negotiation (Application Layer Protocol Negotiation, formerly known as NPN) is used for the Protocol Negotiation between client and server.
2. Http-based negotiation The client can also enable HTTP 2.0 communication using HTTP. Because HTTP 1.0 and HTTP 2.0 both use the same port (80) and there is no other information about whether the server supports HTTP 2.0 or not, the client can only use the HTTP Upgrade mechanism (OkHttp, nGHTTP2, etc.). You can also code it yourself) through coordination to determine the appropriate protocol
3. Complete communication process (1) TCP connection establishment (2) TLS handshake and HTTP2.0 communication process (1) TLS handshake 2) Client sends SETTINGS frame to server, Convention configuration 3) Flow control 4) The client sends HEADERS frames to the server 5) the server returns configuration 6) The DATA frame transmits DATA
Before sending application data, you must create a new stream and send corresponding metadata, such as stream priority, HTTP header, etc. A client sends a HEADERS frame to initiate a new stream. The server initiates a push stream by sending a PUSH_PROMISE frame.
After creating a new stream and sending the HTTP header, the next step is to use the DATA frame. Application DATA can be divided into multiple DATA frames. The END_STREAM field at the end of the frame must be inverted. The HTTP 2.0 standard requires that DATA frames should not exceed 2 ^ 14-1 (16383) bytes. Data that exceeds this threshold is sent as a frame. Using one connection per source, HTTP 2.0 improves performance by maximizing the throughput of a TCP connection. Take advantage of server push: Most resources previously embedded for HTTP 1.x can and should be delivered via server push.
15, HTTPS
15.1 Disadvantages of HTTP
1. The communication is in plain text (not encrypted), and the content may be eavesdropped; TCP/IP is a network that can be eavesdropped on: The way the TCP/IP family of protocols works, communication can be monitored on all lines. Encryption processing to prevent eavesdropping, encrypted objects are as follows:
(1) Encryption of communications Using a combination of SSL(Secure Socket Layer) or TLS(Transport Layer Security) to encrypt HTTP communications. Once a secure communication line is established with SSL, HTTP traffic can be conducted over that line. HTTP used in combination with SSL is called HTTPS. In this way, the entire communication line is encrypted
(2) Content encryption Because there is no encryption mechanism in THE HTTP protocol, the content itself transmitted by the HTTP protocol is encrypted. In order to achieve effective content encryption, both the client and server are required to have encryption and decryption mechanisms. This method is different from the whole communication line encryption processing, the content still has the risk of tampering. 2. The identity of the communicating party is not verified. Therefore, it may be disguised. (1) Anyone can initiate a request. No matter who sends the request through THE HTTP protocol, the response will be returned. Therefore, if the communication party is not confirmed, there will be the following hidden dangers: It is impossible to determine whether the Web server sent to the target of the request is the one that returns the response according to the real intention, or it may be a disguised Web server. It is not possible to determine if the response returned to the client was the one that received the response as intended, possibly a disguised client. It is not possible to determine whether the person you are communicating with has access rights, because some Web servers hold important information and only want to give specific users the right to communicate. There is no way to tell where and from whom a request is coming from. Even meaningless requests are accepted. There is no way to prevent Denial of Service (Dos) attacks with massive requests. SSL not only provides encryption, but also uses a method called a certificate that can be used to identify the communicating party. A certificate is issued by a trusted third-party organization to prove that the server and client exist. The use of certificates to prove that the communicating party is the expected server also reduces the risk of personal information leakage for users. In addition, the client with a certificate can complete the confirmation of personal identity, can also be used for Web site authentication. 3. The integrity of the packet cannot be verified. Therefore, the packet may be tampered with. (1) The received content may be wrong. During the period between the sending of the request or response and the receiving of the request or response, there is no way to know even if the content of the request or response is tampered with. A man-in-the-middle attack is an attack in which an attacker intercepts and tampers with the content of a request or response in transit. (2) How to prevent tampering It is very difficult to ensure integrity only through HTTP, so it depends on HTTPS. SSL provides authentication, encryption, and summary functions.
15.2 How Does HTTPS Encrypt Data
HTTP with added encryption, authentication, and integrity protection is called HTTPS
The predecessor of TLS is SSL
The SSL protocol uses the client certificates and CA root certificates of both parties to allow the client/server to communicate in a way that cannot be overheard, and establishes a secure and trusted communication channel between the two parties.
1. Encryption is divided into symmetric encryption and asymmetric encryption (also called public key encryption)
(1) Symmetric encryption means that the key used to encrypt the data is the same as the key used to decrypt it.
1) The advantage of symmetric encryption is that the efficiency of encryption and decryption is usually higher. Symmetric encryption is usually used when the message sender needs to encrypt a large amount of data. The algorithm is open, the computational amount is small, the encryption speed is fast, and the encryption efficiency is high.
2) The disadvantage is that the data sender and the data receiver need to negotiate and share the same key, and ensure that the key is not disclosed to others. In addition, the cost of distributing and maintaining a key between two individuals with data exchange requirements is basically unacceptable.
(2) Asymmetric encryption means that the key used to encrypt the data (the public key) is different from the key used to decrypt the data (the private key).
A public key is a public key that can be accessed by anyone. A private key is a private key, usually held by the site’s administrator.
The connection between public and private keys: Simply put, the data encrypted with the public key can only be unlocked with the private key. Data encrypted with the private key can only be unlocked with the public key.
2. The public key is faulty
Problem 1: How to obtain the public key;
Problem 2: Data transmission is only one-way security (because the private key encrypts the data and the public key can unlock it, so it is only one-way security)
(1) How to obtain the public key
There are two important concepts involved: certificates and CA (Certificate Authority)
Certificate: can temporarily understand it as the identity card of the website. This ID contains a lot of information, including the public key mentioned above. When the site is visited, he sends the certificate to the browser;
The certificate comes from the CA (Certificate Authority)
(2) Possible problems with the certificate:
The certificate is forged: the certificate is not issued by the CA at all. It has been tampered with: for example, the public key of the XX website has been replaced
Digital signature and summary are the key weapons of certificate security. Abstract is a string of fixed length calculated by the hash algorithm on the transmitted content. The digest is then encrypted through the CA’s private key, and the result is a “digital signature.” Plaintext -> Hash operation -> Digest -> Private key encryption -> Digital signature Digital signature only the CA public key can decrypt. Certificate format, need to pay attention to the following points:
The certificate contains the name of the authority that issues the certificate, the digital signature of the CA certificate itself (encrypted with the CA private key), and the hash algorithm used by the certificate holder to sign the public key certificate
The CA has its own certificate, called the root certificate. 1) Check the certificate in case of completely forged certificate
The certificate issuing authority is forged: the browser does not recognize the certificate and directly considers it to be a dangerous certificate. The certificate issuing authority does exist. Therefore, it finds the built-in CA root certificate and the CA public key according to the CA name. The abstract of the forged certificate was decrypted using the CA public key, but could not be solved. Considered dangerous certificate
2) Tampered certificates
Check the certificate and find the CA root certificate and the CA public key according to the CA name. The public key of CA is used to decrypt the digital signature of the certificate, and the corresponding summary of the certificate is obtained. AA calculates the summary BB of the current certificate according to the hash algorithm used by the certificate signature, compares AA and BB, and finds the inconsistency –> determines that the certificate is dangerous
15.3 the HTTPS process
1. The client initiates an HTTPS request
2. The server responds and delivers the certificate (public key certificate).
3. The client checks the certificate. If the certificate is valid, it generates a random value and encrypts the random value using the certificate (public key).
4. Send the random value encrypted by the public key to the server (asymmetric encryption). Then the communication between the client and the server can be encrypted and decrypted using the random value.
5. After decrypting with the private key, the server obtains the random value sent by the client, and then performs symmetric encryption through the value.
6. Later data transmission is encrypted and decrypted based on the random value.
15.4 The Problem of obtaining public keys
Possible problems concerning certificates and CA (Certificate Authority) certificates are as follows: 1. The certificate is forged and not issued by the CA. 2. 2. The certificate is tampered With The digital signature and digest are the key weapons for certificate security plaintext –> Hash algorithm –> digest —-> Private key encryption —–> Digital signature Digital signature Only the CA public key can decrypt the CA certificate has its own certificate, called the root certificate. (1) Check the certificate for the completely forged certificate
The certificate issuing authority is forged: the browser does not recognize the certificate and directly considers it to be a dangerous certificate. The certificate issuing authority does exist. Therefore, it finds the built-in CA root certificate and the CA public key according to the CA name. Using the CA public key to decrypt the digital signature of the forged certificate, it was found that it could not be solved. Considered dangerous certificate
(2) Tampered certificates
Check the certificate and find the CA root certificate and the CA public key according to the CA name. The public key of CA is used to decrypt the digital signature of the certificate, and the corresponding summary of the certificate is obtained. AA calculates the summary BB of the current certificate according to the hash algorithm used by the certificate signature, compares AA and BB, and finds the inconsistency –> determines that the certificate is dangerous
15.5 Client Certificate
The client certificate can also be used in HTTPS for client authentication, which proves that the client that the server is communicating with is always an expected client. To obtain the certificate, you need to install the client certificate. As it is, highly secure certification authorities can issue client certificates but only for special purposes, such as online banking.
15.6 the HTTPS problem
1. Compared with plain text, encrypted communication consumes more CPU and memory resources. HTTPS requires encryption and decryption on both the server and client, which consumes hardware resources such as CPU and memory. Compared with HTTP communication, SSL communication consumes network resources, and SSL communication takes longer time because it has to process the communication. There are two types of SSL slow. One is slow communication. The other is that the processing speed slows down due to the high consumption of CPU and memory resources. 2. It costs to purchase a certificate.
16, WebSocket
16.1 the WebSocket API
1.WS and WSS WebSocket resource URLS adopt custom mode, WS means plain text communication, WSS means using encrypted channel communication (TCP+TLS)
The main purpose of WebSocket is to provide an optimized bidirectional communication mechanism between the application in the browser and the server. However, the WebSocket connection protocol can also be used in scenarios outside the browser where data can be exchanged through non-HTTP negotiation mechanisms.
2. Receive text and binary data
WebSocket communication is all about messages, and the application code doesn’t have to worry about caching, parsing, and rebuilding the data it receives. For example, if the server sends a 1MB load, the application’s onMessage callback is only called when the client has received all the data.
The net load can be either text or binary data.
When the browser receives a new message, it automatically converts it to a DOMString object if it is text data, or to a Blob object if it is binary data (a Blob object typically represents an immutable file object or raw data). This format is ideal if you don’t need to modify it or cut it into smaller chunks (ArrayBuffer is more appropriate if you need to process it) and will be passed directly to the application. The only unnecessary setting is to tell the browser to convert the binary data received into an ArrayBuffer rather than a Blob
3. Send text and binary data
The client can then send or receive UTF-8 or binary messages at any time
The send() method here is asynchronous: the data provided is queued on the client side, and the function returns immediately. Especially when transferring large files, do not assume that the data has been sent just because it returns quickly. The bufferedAmount property allows you to monitor the amount of data queued in the browser. ws.bufferedAmount
4. Subprotocol negotiation
The WebSocket protocol makes no assumptions about the format of each message: just one bit marks whether the message is text or binary so that the client and server can effectively decode the data, but beyond that the content of the message is unknown.
Furthermore, unlike HTTP or XHR requests, which communicate metadata through the HTTP headers of each request and response, WebSocket does not have an equivalent mechanism. Therefore, if metadata about a message needs to be communicated, the client and server must agree on a subprotocol to communicate this data.
WebSocket provides a simple and convenient subprotocol negotiation API for this purpose. The client can tell the server which protocol it supports during the initial connection handshake:
If the subprotocol negotiation succeeds, the onOpen callback on the client is triggered. The application can query the protocol attribute on the WebSocket object and know the protocol selected by the server. On the other hand, if the server does not support any of the protocols declared by the client, the WebSocket handshake is incomplete, an onError callback is triggered, and the connection is disconnected.
16.2 the WebSocket protocol
The WebSocket communication protocol consists of two high-level components: an open HTTP handshake to negotiate connection parameters, and a binary message framing mechanism to support low-overhead message-based text and binary data transmission.
1. Binary frame layer
WebSocket uses a custom binary frame splitting format, which cuts each message into one or more frames, sends them to the destination and then asseminates them, and notifies the receiver when the complete message is received.
Message: a series of frames
FIN: Indicates whether the current frame is the last frame of the message.
Opcode (4 bits) : Indicates the type of frame being transmitted: text (1), binary (2)
The mask bit indicates whether the net load has a mask (only applicable to messages sent by the client to the server).
Each WebSocket frame sent by the server will generate 210 bytes of frame overhead. The client has to send the mask key, which adds another 4 bytes, resulting in 614 bytes of overhead. Other than that, there are no other elements.
2.WebSocket multiplexing and queue head blocking
Websockets are prone to queue-first blocking: messages may be split into one or more frames, but frames from different messages cannot be interlaced because there is no field equivalent to the “stream ID” in HTTP2.0’s frame-splitting mechanism.
WebSocket does not support multiplexing, which also means that each WebSocket connection requires a dedicated TCP connection. With HTTP1.x, this can cause problems because the browser has a limit on the number of connections per source.
Although the official specification for transmitting WebSocket frames over HTTP2.0 has not yet been released, it is relatively easy. Because HTTP2.0 has built-in multiplexing of streams, multiple WebSokcet connections can be transmitted in a single session as long as the WebSocket frames are encapsulated using HTTP2.0’s frame-splitting mechanism.
3. Protocol extension
(1) Multiplexing extension
This extension can isolate WebSocket logical connections and share underlying TCP connections.
(2) Compression extension
Added compression to the WebSocket protocol.
As mentioned earlier, each WebSocket connection requires a dedicated TCP connection, which is inefficient. The multiplexing extension solves this problem. It uses a “channel ID” to extend each WebSocket frame so that multiple virtual WebSocket channels share a TCP connection. Similarly, the basic WebSocket specification has no mechanism or recommendation for compressing data, and the net load in each frame is the net load provided by the application. While this is not a problem for optimized binary data structures, unless the application implements its own compression and decompression logic, it can in many cases result in excessive transmission loads. In effect, compression extensions are the equivalent of HTTP’s transports encoding negotiation.
To use an extension, the client must notify the server in the first Upgrade handshake, and the server must select and confirm the extension to use in the negotiated connection.
4.HTTP upgrade negotiation
Before exchanging data, the client must negotiate the appropriate parameters with the server to establish a connection.
There are several benefits to using HTTP to complete the handshake. First, make WebSocket compatible with the existing HTTP infrastructure: WebSocket servers can run on ports 80 and 443, which are usually the only ports open to clients. Second, let’s reuse and extend the HTTP Upgrade stream by adding custom WebSockets to it
Header to complete the negotiation.
16.3 WebSocket Usage Scenarios and Performance
1. Request and Response Flow WebSocket is the only mechanism that enables two-way communication over the same TCP connection. Client and server can exchange data at any time. 2. Message overhead Application messages are split into one or more frames and each frame adds an overhead of 2 14 bytes. Furthermore, because framing is done in a custom binary format, UTF-8 and binary application data can effectively be encoded through the same mechanism. (1) SSE adds 5 bytes to each message, but only for UTF-8 content (2) HTTP 1.x requests (XHR and other general requests) carry 500, 800 bytes of HTTP metadata, plus cookies (3) HTTP 2.0 compression HTTP metadata, This can significantly reduce overhead 3. Data efficiency and compression Unless the application implements its own compression logic by carefully optimizing its binary net load, and also implements its own compression logic for text messages, the data transmission process is bound to incur a large byte overhead! 4. Customize application protocols
16.4 Diagram HTTP explanation
WebSocket technology is mainly designed to solve the problems caused by the flaws attached to XMLHttpRequest in Ajax and Comet. WebSocket is a protocol based on HTTP, so the initiator of the connection is still the client. Websocket requires only one HTTP handshake, so the entire communication is established in one connection/state, which avoids the non-state nature of HTTP. The server will always know your information. Until you turn off requests. (1) WebSocket’s main features. (1) The push function enables the server to push data to the client. 2) Reduce traffic As long as a WebSocket connection is established, you want to stay connected. Compared with HTTP, not only the total overhead of each connection is reduced, but also the traffic is correspondingly reduced because the header information of WebSokcet is small. (2) Handshake Request In order to realize WebSocket communication, after the HTTP connection is established, a “handshake” step needs to be completed. To implement WebSocket communication, you need to use the Upgrade header field of HTTP to inform the server that the communication protocol has changed to achieve the purpose of handshake.
Sec-websocket-key: records the Key value required for the handshake. Sec-websocket-protocol: records the subprotocols used.
(3) Handshake Response To the previous request, returns a status code 101 Switching Protocols response. After the successful handshake establishes the WebSocket connection, the communication does not use HTTP data frames, but uses WebSockte independent data frames;
Sec-websocket-accept: The value of this field is generated by the value of the sec-websocket-key field in the handshake request.
16.5 Advantages of WebSocket (Comparing with HTTP)
Yq.aliyun.com/articles/63… (1) Support two-way communication, more real-time; (2) Better binary support; (3) Less control overhead: After the connection is created, when the WS client and server exchange data, the packet header controlled by the protocol is smaller. Without headers, the server to client header is only 2 to 10 bytes (depending on the packet length). Client to server requires an additional 4 bytes mask. HTTP requires a complete header for each communication; (4) Support for extensions: The WS protocol defines extensions. Users can extend the protocol or implement custom sub-protocols (such as support for custom compression algorithms, etc.).
16.6 How Do I Set up a Connection
WebSocket multiplexes the HTTP handshake channel. Specifically, the client negotiates the upgrade protocol with the WebSocket server through HTTP requests. After the protocol upgrade is complete, the subsequent data exchange follows the WebSocket protocol. 1. Client: Applies for a protocol upgrade. First, the client initiates a protocol upgrade request. As you can see, the standard HTTP packet format is used and only the GET method is supported. Connection: Upgrade: indicates the protocol to be upgraded. Upgrade: websocket: indicates the webSocket protocol to be upgraded. Sec-websocket-version: 13: indicates the WebSocket Version. If the server does not support the version, return a Sec- Websocket-VersionHeader containing the version number supported by the server. Sec-websocket-key: Is compatible with the sec-Websocket-Accept in the response header of the following server and provides basic protection against malicious or unintentional connections. 2. Server: Responds to the protocol upgrade. The return status code 101 indicates the protocol switchover
16.7 Data Frame Format
The data exchange between client and server is inseparable from the definition of data frame format. The minimum unit of communication between WebSocket client and server is frame, which consists of one or more frames to form a complete message. The sender: cuts the message into multiple frames and sends them to the server. Receiver: Receives a message frame and reassembles the associated frames into a complete message. See the above two summaries for frame content
16.8 Data Transfer
Once the WebSocket client and server have established a connection, subsequent operations are based on the transfer of data frames. WebSocket distinguishes the types of operations based on opCode. For example, 0x8 indicates disconnection, and 0x0-0x2 indicates data interaction. 1. Data fragmentation Each WebSocket message may be divided into multiple data frames. When a WebSocket receiver receives a data frame, it uses the FIN value to determine whether the last data frame of the message has been received. FIN=1 indicates that the current data frame is the last data frame of the message. At this time, the receiver has received the complete message and can process the message. If FIN=0, the receiver needs to continue listening for the remaining data frames. In data exchange scenarios, opCode indicates the data type. 0x01 indicates text and 0x02 indicates binary. In particular, 0x00 represents a continuation frame, which, as the name implies, is a data frame for a complete message that has not yet been received.
16.9 Connection Maintain + Heartbeat
WebSocket To maintain real-time bidirectional communication between the client and server, ensure that the TCP channel connection between the client and server is not disconnected. In some scenarios, the connection between the client and the server needs to be maintained even though there is no data exchange between the client and the server for a long time. In this case, you can use heartbeat to implement the following operations: Sender -> Receiver: Ping the receiver -> sender: Pong The operation of ping and pong corresponds to two WebSocket control frames with opcode 0x9 and 0xA. For example, ws.ping(“, false, true);
16.10 Sec- websocket-key /Accept Functions
The main role of sec-websocket-key/sec-websocket-Accept is to provide basic protection and reduce malicious and unexpected connections.
16.11 Functions of the Data mask
In WebSocket protocol, the function of data mask is to enhance the security of the protocol. But data masks are not intended to protect the data itself (since the algorithms themselves are public and uncomplicated), but to prevent problems such as proxy cache contamination attacks that existed in earlier versions of the protocol.
17. Differences between status codes 301 and 302
(1) What is a 301 redirect? 301 redirect/jump general, indicates that this page permanently transferred to another address. 301 is permanent move (Permanently Moved), SEO (search engine optimization) commonly used move, will move the old page PR (permanent residence) and other information to the new page; (2) What is a 302 redirect? 302 Temporarily Moved, and used when a web page URL needs to be changed for a short time. (3) 301 redirect and 302 redirect difference 301 redirect is a permanent redirect, search engines in the crawl of new content at the same time will replace the old url after the redirect url. 302 redirects are temporary redirects in which search engines crawl new content while retaining old urls. Because the server returns a 302 code, the search engine thinks the new url is temporary.
Xviii. Solution to the problem caused by strong caching (how to update site resources)
Problem: Resource update problem at release time. The resource is updated, but the original resource is still obtained from the cache every time the user requests it. Unless the user clears or forcibly refreshes, the latest effect cannot be seen. 1, use 304 redirect redirect, let the browser local cache is the negotiation cache. Cons: You still need to communicate with the browser once to negotiate the cache. 2. Force the browser to use the local cache and do not communicate with the server. Disadvantages: Cannot update cache. 3. Update the resource path referenced in the page, so that the browser gives up the cache actively and loads new resources. For example, add a version number to the end of the file name and change the version number the next time the file goes online. Disadvantages: When there are multiple statically cached files, all need to be updated when only one file changes.
Data summary algorithm is adopted
To solve this problem, consider making url changes associated with the contents of the file, i.e. only when the contents of the file change, will the corresponding URL change, thus achieving precise cache control at the file level. To further improve website performance, static resources and dynamic web pages are deployed in clusters. Static resources are deployed to CDN nodes, and resources referenced in web pages become corresponding deployment paths. When I update the static resource, I also update the reference in the HTML. If this release, at the same time changed the page structure and style, also updated the url address corresponding to static resources, now to release the code online, we are the first online page, or first online static resources?
Deploy the page first, then deploy the resource: If a user visits the page between the two deployments, the old resource is loaded in the new page structure, and the old version of the resource is cached as the new version. The result is: The user is accessing an unstyled page that will continue to perform errors until the resource cache expires unless manually refreshed. Deploy resources first and then pages: During the deployment period, if a user with the local cache of resources of an old version visits a website, the browser directly uses the local cache because the requested page is of an old version and the resource reference has not changed. In this case, the page is displayed properly. However, when users who have no local cache or whose cache is expired visit the website, the page of the old version will load resources of the new version, resulting in a page execution error. However, when the page is deployed, these users will return to normal when they revisit the page.
Any of the above is problematic because of the overwriting publishing, where published resources are overwritten with resources to be released. The solution is to implement non-overridden publishing. Static resource optimization for large companies: 1. Configure the local cache for a long time to save bandwidth and improve performance 2. Use the content summary as the basis for cache update — accurate cache control 3. Deploy the static resource CDN — optimize the network request 4. Update resource publishing path to achieve non-overwriting publishing — smooth upgrade
Nineteen, sliding window principle
19.1 TCP Sliding Window (Sending window and Receiving Window)
The TCP sliding window has two functions: one is to provide TCP reliability, and the other is to provide TCP flow control characteristics (TCP sliding window is dynamic). At the same time, the sliding window mechanism also reflects the design idea of TCP for byte stream.
TCP Window is a 16bit field, which represents the Window capacity in bytes. That is, the maximum size of a standard TCP Window is 2^ 16-1 = 65535 bytes.
1. For the sender of a TCP session, the data in the send cache at any time can be classified into four categories: “Sent and received an ACK from the peer”, “Sent but not received an ACK from the peer”, “not sent but allowed to be sent by the peer”, and “Not sent but not allowed to be sent by the peer”. The data that has been sent but has not received an ACK from the peer end and the data that has not been sent but is allowed to be sent by the peer end are called the sending window (the middle two parts).
2. For the TCP receiver, there are three types of memory in the receive cache at a certain time. “Received,” “Not received ready to receive,” “Not received not ready to receive. Unreceived ready to receive is called the receive window.
3. Relationship between the sending window and the receiving window
TCP is a duplex protocol. Both sides of a session can receive and send data at the same time. Each side of a TCP session maintains a send window and a receive window. The size of each “receive window” depends on the limitations of the application, system, and hardware (the TCP transmission rate cannot be greater than the data processing rate of the application). The requirements for the various “send Windows” depend on the “receive Windows” advertised by the peer end, and the requirements are the same.
4. Sliding Windows achieve stream-oriented reliability
The most basic transmission reliability comes from the “confirm retransmission” mechanism.
The reliability of TCP’s sliding window is also based on confirmatory retransmission.
The sending window will move the left margin of the sending window only after receiving the ACK acknowledgement of the byte in the sending window from the peer.
The receive window will only move the left margin if all previous segments are confirmed. The window does not move and does not acknowledge subsequent bytes when the preceding byte is not received but the following byte is received. In this way, the peer end can retransmit the data.
19.2 Classification of TCP Network protocols
At present, there are many network protocols built on THE TCP protocol, such as Telnet, SSH, FTP, HTTP and so on. These protocols can be roughly divided into two categories according to the data throughput: (1) Interactive data type, such as Telnet, SSH, this type of protocol in most cases only do small traffic data exchange, such as pressing the keyboard, echo some text and so on. (2) Data block type, such as FTP. This type of protocol requires TCP to carry data as much as possible, maximize data throughput and improve efficiency as much as possible.
19.3 Terms related to the sliding window protocol
A sliding window is essentially a piece of data that describes the size of the TCP datagram buffer on the receiving side, which the sender uses to calculate the maximum length of data it can send. If the sender sends a TCP datagram with a window size of 0 to the receiver, the sender will stop sending data until the receiver sends a datagram with a non-0 window size. Three terms are introduced for the sliding window protocol, which are:
Window closing: When the window approaches from left to right, this occurs when data is being sent and confirmed. Window opening: When the right edge of the window moves to the right after the data has been processed at the receiving end. Window shrinkage: When the right side of the window moves to the left, this does not happen very often
19.4 Take an example to illustrate the principle of sliding Windows
TCP does not respond to each packet segment with an ACK. It may send one ACK for two packet segments or one ACK for multiple packet segments. For example, the sender has 1, 2,3 packet segments and sends 2,3 packet segments first, but the receiver expects to receive 1 packet segment. If packet 1 does not arrive, 2/3 of the packet will be discarded. If packet 1 does arrive, an ACK will be sent to acknowledge the three packets once.
Here’s an example of how sliding Windows work:
1. The hypothesis of 3245 The upper-layer Application sends the data to TCP. TCP divides the data into four segments and sends it to the Internet34 seg3 35
2. seg1 3236 seg3 3741 seg4 4245 The four fragments are sent out in turn, assuming that the receiver has received seG1 seG2 Seg436, and the SEG4 is cached (in order to generate a hole for seG3).
3. The receiver responds with an ACK packet indicating that 32 has been received
4. After receiving ACK, the sender switches packets of 32 to 36 from unacknowledged to acknowledged, and raises the window. At this time, the window moves to the right
5. Assume that the Window Size advertised by the receiver remains the same, and then the Window moves to the right, generating some new vacancies. These are the categories that the receiver is allowed to send
6. For the lost SEG3, if the retransmission time exceeds a certain period, TCP will retransmit (retransmission mechanism). If the retransmission succeeds, seG3 SEG4 will be confirmed
In fact, the notification of the Window Size of the receiver will also change. According to this value, the receiver will determine when and how much data to send and control the flow of the data stream. The schematic diagram is as follows:
19.5 Dynamic Adjustment on a Sliding Window
Mainly according to the reception of the receiving end, dynamic to adjust the Window Size, and then to control the data flow of the sending end
The client continues to send data rapidly, while the server receives data relatively slowly. Look at the results of the experiment
A. Packet 175, send ACK with WIN = 384, inform the client that only 384 bytes can be received now
B. Packet 176. The client sends only 384 bytes
C. 177, the server reply an ACK, and circular window is zero, the receiver has received all the data, and save to the buffer, but this time the application did not receive these data, the buffer is no more space, so the notification window is zero, the window is zero, zero during the window, the sender to stop sending data
D. When the client detects that the window is 0, it no longer sends data to the receiver
E. Packet 178. The receiver sends a window notification informing the sender that it has the ability to receive data and is ready to send the packet
F. Packet 179, after receiving the window notification, the data in the buffer is sent.
From Memory Cache and From Disk Cache in Chrome
1. Three situations occur in the Size column of Network in the browser developer tool
- from memory cache
- from disk cache
- Size of the resource itself (e.g. 13.6k)
2. Level-three cache principle
First look for memory, if there is memory, load from memory; If it is not found in the memory, select the hard disk to obtain it. If it is found in the hard disk, load it from the hard disk. If it is not found on the hard drive, the network request is made. Loaded resources are cached to hard disk and memory;
3. HTTP status codes and their differences
- 200 form memory cache
Do not access the server. Generally, the resource is loaded and cached in memory, and the cache is read directly from memory. When the browser is closed, the data will not exist (the resource has been released), and when the same page is opened again, the from memory cache will not appear.
- 200 from disk cache
After the browser is closed, the data still exists. The resource will not be released when the page is closed. The next time the page is opened, the data will still be from disk cache.
- 200 size (e.g. 3.4K)
When accessing the server, size is displayed as the size of the actual resource
- 304 Not Modified
The server returns this status code when the data has not been updated. The data is then read from the cache. This has two request parameters in the request header: if-modified-since and if-none-match.
| Status code | type | instructions |
|---|---|---|
| 200 | form memory cache | Network resources are not requested, they are in memory |
| 200 | form disk ceche | Do not request network resources, in disk |
| 200 | Resource size value | Download the latest resources from the server |
| 304 | Packet size | The requesting server finds that the resource has not been updated and uses the local resource |
These are two of the most common HTTP status codes chrome uses to request resources, so stylesheets are usually on disk, not cached in memory, because CSS styles can be loaded once to render a web page. However, the script may be executed at any time. If the script is on disk, the execution of the script needs to be fetched from disk to memory, which is a large IO overhead and may cause the browser to lose response.
Welcome to follow the public account (reply “nginx” to get the mind map of this section, reply “books” to get a lot of front-end learning materials, reply “front-end video” to get a lot of front-end teaching videos)