preface

This paper will continue to record the principles of HTTP and TCP/IP mastered by the author in the learning process, as well as some application scenarios of these network communication technologies. The paper will keep updated, which is equivalent to a summary and induction of this knowledge. Have incorrect place welcome to point out, correct in time ~

The outline

  • What happens when you visit a web page
  • Classical five-layer network model
  • Uris, urls and UrNs
  • TCP/IP protocol family
  • A TCP connection
  • TCP’s three handshakes and four waves
  • HTTP defines
  • HTTP history
  • HTTP packets
  • HTTP Data negotiation
  • Long HTTP connection
  • HTTP cache
  • The same origin policy of the browser
  • HTTP performance optimization solution

What happens when you visit a web page

When a user enters an address in the browser’s address bar and hits Enter, it takes about two or three seconds to see the web interface. In that instant, however, the computer has actually performed a very complex operation. A large part of what happens in this process is actually related to HTTP TCP/IP, and we can briefly outline the process.

The first step is to find the server IP

When a user types a web address and presses enter, the browser gets a domain name. For actual communication, the browser needs an IP address. To obtain an IP address, the browser performs the following operations. The behavior of the browser to search for an IP address by domain name is called DNS resolution.

  1. Look for the browser’s local cache first
  2. Look for the one on the hard drivehostFile, there is no record of this domain name andIPMapping relation of
  3. Really did not find, had to go through the network link domain name supplier there query

Step 2: Establish a TCP/IP connection

  1. The browser gets the serverIPTheta will correspond to thetaIPThe server sendsTCPConnection request.
  2. The server responds to the request, and the two parties confirm it several timesTCP bidirectional connection.

The process from the client initiating a connection request until the TCP connection is established is called the three-way handshake.

If the request is HTTPS, you need to use SSL or TLS to encrypt data, verify the identity of the peer, and ensure data integrity over the TCP connection to ensure data transmission security.

Step 3: Request resources

  1. TCPWhen the connection is created, the browser starts sending formal messages to the serverHTTPRequested packets.
  2. The server receives the request, parses the request, and returns the data packet required by the client after data operation.

Step 4, browser render

After obtaining the required data, the browser splices, analyzes and executes the data, and eventually draws the complete web page on the page.

Step 5: Browser cache

After receiving the data returned by the server, the browser caches the data according to a certain policy. In this way, the next time the browser requests the same data, it directly goes to the cache and does not request the server.

The above process can be regarded as a practical scenario applied in the process of complete network communication, which brings out a lot of knowledge points of network communication. The following takes this line as the index to elaborate and explain the knowledge fragments involved in it.

Classical five-layer network model

In each computer equipment, there is such a set of system link relationship to ensure the normal network transmission, because the unified integration of such a set of classical model, so their use of the computer can also serve as a server to provide network services.

Application layer:

The application layer contains the HTTP protocol, which provides many services for various application software. Common application layer services include HTTP, FTP, Email and so on. The application layer hides the details of the underlying model and, as application support, provides users with only the necessary ways to use it.

The transport layer

Common transport layer protocols include TCP and UDP. Based on the application layer, the transport layer defines the end-to-end data transmission mode. For example, how to establish a connection between two devices? What specifications are required for data transfer between devices? In what way does the data need to be sharded, reassembled, and spliced? These are all defined for us by the transport layer.

The network layer

Usually the IP protocol is located at this layer. The network layer creates logical links for data transmission between nodes. When we type the domain name in the browser, how the browser finds the corresponding IP mapping through the domain name in the network, and the logical relationship and link of this query are standardized and defined by the network layer.

Data link layer

Data link layer establishes data link connection between communication entities. After the physical device connection is completed, corresponding software and driver are required to connect and get through these physical devices to create circuit connection.

The physical layer

Define how physical devices transmit data. Common physical layers include network cables, optical cables, network cards, sound cards, etc. The physical layer is the foundation of all software.

Uris, urls and UrNs

While we are familiar with urls, we may not know much about URIs and UrNs, which are standard ways to identify, locate, and name resources on the Internet.

The moment we type a domain name into the browser address bar, we are already connected to all three concepts.

URI

  • A Uniform Resource Identifier is a Uniform Resource IdentifierURI.
  • Each Web server has oneURIIdentifier, which uniquely identifies and locates information resources around the world, a resource information hasURIOnce identified, the resource can be accessed from a fixed address on the Internet.
  • It comes in two forms, URN (Uniform resource name) and URL (uniform resource locator), that isURL 和 URNIt’s a subset of that.

URL

Uniform Resource Locator (URL). The following figure shows a Uniform Resource Locator (URL).

A complete URL contains the following sections from left to right:

  1. schemaIdentifies the access protocol on which the resource address is based. Common examples are:HTTP 和 FTP.
  2. User Information identifies user information (if the resource requires user information authentication), which is generally not done today because it is cumbersome to enter and insecure.
  3. hostIdentifies the domain information of the resource, which can be a domain name or domain nameIPTo find the address of the physical server where the resource is stored.
  4. Port Indicates the port number of a physical server. By enabling different ports, multiple Web servers can run at the same time. Resource files are deployed on a certain web server, and the port number is used to locate the web server where the resources exist.
  5. Path A Web server has many directories. The path is used to locate the directory where the resource files are stored. Since many Web applications are so large these days, this path is not necessarily a directory address, but may also be a request address for a static resource file specified by the Web server.
  6. queryQuery string, generally used forGETQuery, pass query parameters.
  7. Fragments, hashes, or anchors, are mainly used to locate front-end documents, or to control routing during front-end rendering.

It is important to distinguish urls from web addresses. URL contains not only the resource address of the web page, but also the hypertext resources such as pictures and videos required to form the web page, as well as the resource address of CSS AND JS. A URL is essentially a more discernable mapping of an IP address. After the DNS resolution, the browser first gets the URL of an HTML document. According to the browser’s resolution of the HTML document, the browser continues to obtain the corresponding resource file from the URL of other resource files in the web page.

URN

Uniform Resource Name, or URN, solves this problem by permanently locating the Resource because the same Resource may change its storage location. Once the storage location is changed, you cannot access the original URL again. Accessing the same URN will locate the resource no matter how it is moved.

TCP/IP protocol family

The establishment of TCP connection is an indispensable step for successful network communication. Therefore, TCP/IP has become a very important knowledge point.

TCP/IP (Transmission Control Protocol/Internet Protocol) is not a single protocol, but a group of special protocols, including TCP, IP, UDP, ARP, etc. These are called sub-protocols. Of these protocols, the most important and well-known are TCP and IP. So we tend to refer to the entire protocol family as TCP/IP.

  • TCP/IP protocol
    • IPProtocols make the Internet a network that allows connections between different types of computers and different operating systems.
    • IPThe address isIPA unified address format that assigns a logical address to each network and each host on the Internet. It is equivalent to the temporary name of the machine. Other machines can use this name to find it and establish a connection to communicate with each other.
  • TCP protocol
    • TCPThe protocol is a connection-oriented full-duplex protocol, so both the client and the server can be used in theTCPThe connection channel receives and sends data to the peer end.
    • TCPCompared to theUDPThe advantage lies in its transmission stability, before data transmission must be through three handshakes to establish a connection; Ensure that data is transferred to the peer end in an orderly and complete manner.
    • TCPCompared to theUDPThe disadvantage is its complexity, connection establishment and disconnection are relatively high performance overhead, and once the data transmission process is stuck, it must wait for the previous data to be sent before the subsequent data can continue to be transmitted.
    • Each server is available for supportTCPThe number of connections is finite, so this also makesTCPConnections become scarce resources that cannot afford to be wasted.
  • UDP protocol.
    • UDPThe protocol is connectionless and does not require a connection to be established before data can be transmitted.
    • UDPPackets are transferred to the peer end in an orderly manner without loss. Therefore, packet loss may occur.
    • UDPSupport not only one-to-one transmission mode, but also support one-to-many, many-to-many, many-to-one mode, that is to sayUPDProvides unicast, multicast, broadcast functions.
    • UDPCompared to theTCPThe advantage is that it is lightweight, efficient and flexible, and needs to be used in some scenarios with high requirements for real-time applicationUDP, such as live streaming, video conferencing, LOL and other real-time battle games.
    • UDPCompared to theTCPThe disadvantage lies in its unreliability and instability.

A TCP connection

Before a client sends an HTTP request, you need to create a TCP connection. In this way, all HTTP requests and responses can be sent and received properly.

The creation and persistence mechanism for this TCP connection channel varies between HTTP versions.

  • inHTTP1.0Every timeHTTPEach request creates oneTCPConnect, after the request has been sent and the server has responded, thisTCPThe connection is automatically disconnected.
  • inHTTP1.1, you can set it manuallyConnection： keep-aliveRequest headers to buildTCPPersistent connection, multipleHTTPRequests can share oneTCPThe connection. butTCPThe connection is blocked, that is, several requests are queued to be sent. Once a request times out, the subsequent requests are blocked.
  • inHTTP2, using channel multiplexing, so thatTCPConnections support concurrent requests, that is, multiple requests can be executed concurrently on a connection. One request task is too time-consuming to affect the normal execution of other connections. In this way, most requests can use oneTCPConnect instead of creating a new oneTCPThe connection channel saves the cost of three-way handshake and server maintenanceTCPCost of ports.

TCP’s three handshakes and four waves

Three-way handshake

Tip: About the meanings of ACK, FIN, and SYN status codes

  1. ACKUsed to confirm that I have received your message.
  2. FINIt is used to terminate the connection to inform the other party that the connection has been terminated, all data has been sent and there is no follow-up output.
  3. SYNUsed to synchronize and establish a connection, indicating that the other party, I request synchronous establishment of a connection.
  1. First handshake: the client sends a connection request to the serverSYNA packet containing its initial data communication sequence number. After the request is sent, the client enters the packetSYN-SENTState.
  2. Second handshake: after receiving the connection request packet segment, if the server agrees to connect,Will send a containingACK 和 SYNA response to a message, which also contains its own initial data communication sequence number(During the “four waves” of disconnection,ACK 和 SYNThe two packets are sent separately as two replies, so there are four waves.) The server enters after sending the packetsSYN-RECEIVEDState.
  3. Third handshake: When the client receives a connection approval reply, it sends an acknowledgement packet to the server. The client enters after sending this packet segmentESTABLISHEDState, which the server enters upon receiving the replyESTABLISHEDThe connection is successfully established.

One of the questions you might ask in an interview is, why do you need three handshakes when two can confirm a connection? Because many uncontrollable factors, such as network, may cause the first request to reach the server after a long time. At this time, the client has waited for the response for a long time. The previous request has timed out and is discarded by the client and no longer listens. However, after a long time, the server receives the deprecated delay request, responds with a new TCP connection port, and waits for the client. The number of TCP connections that the server can maintain is limited. Such idle connections waste resources on the server. Therefore, after the server sends the SYN and ACK responses, it needs to receive the acknowledgement from the client to establish a connection. The three-way handshake is designed to avoid the server overhead caused by network latency.

Four times to wave

Similar to establishing a TCP connection, disconnecting a TCP connection requires bidirectional communication between the client and the server. The disconnecting action requires four packets sent by both ends to complete, so it is called “four waves” for short.

  1. First wave: the client thinks it has sent all its data and sends oneFINIt is used to close the data transfer from the client to the server. After the data transfer is complete, the client entersFIN_WAIT_1State.
  2. Second wave: the server receives the message from the clientFINLater, the application layer is told to release the TCP connection and send oneACKTo the client, indicates that it has received the release request from the client and will not accept the data sent by the client. Then, the server entersCLOSE_WAITIn the state.
  3. Third wave: the server can continue to send data if it has not finished sending. After sending data, the server will also send a releaseFINThe request is used to close the server-to-client data transfer, and then the server entersLAST_ACKState.
  4. Fourth wave: The client receives the packet from the serverFINAfter the request, send the last oneACKTo the server, and then inTIME_WAIT_2State, which lasts for 2MSL (maximum segment lifetime). If the server does not resend the request within this period, the client will enterCLOSEDStatus. After receiving the reply message, the server also entersCLOSEDAt this point, the four waves are completed and the two sides are officially disconnected.

This may not be intuitive, so I’ve prepared a personal statement to describe the process:

  1. Client: Hello, I’m ready.
  2. Server: Oh, are you ready? I see. I’m not ready.
  3. Server: OK, I’m fine now.
  4. Client: Got it. We had a great time. We’ll make an appointment next time.

You might ask in an interview, why are there three handshakes to establish a connection and four handshakes to disconnect? This is because during connection establishment, the server sends an ACK and A SYN packet to the client after receiving a SYN packet from the client. And close the connection, the server received the client’s FIN message, saying only that the client no longer send data, but also can receive data, and the server at the moment there may be data is not sent out, can’t send the FIN immediately message, only an ACK packet first, in response to the client first, after to make sure all data is sent their side, The FIN is sent. As a result, the ACK and FIN of the server are typically sent separately on disconnection, resulting in one more send for disconnection than for connection request.

HTTP defines

Once the TCP connection has been successfully established end-to-end, the next step is to send a formal HTTP request. HTTP, which flows through the TCP Connect channel, is only responsible for transmitting data packets and does not have the concept of connection. Therefore, HTTP is also called “stateless protocol”.

HTTP is the abbreviation for Hyper Text Transfer Protocol. It usually runs on TOP of TCP. It interacts with data through browsers and servers and transfers Hyper Text, images, and videos. That is, the HTTP protocol dictates the rules to be followed for hypertext transmission.

  1. HTTPThe protocol is stateless. This means that the client and the server have no information about each other’s current state,HTTPThe request itself is not stored with any state. In practice, however, client and server must require state authentication and interaction, so it is introducedCookie, used to store some state information of the current browser, each time through a separateHTTPRequests are sent and received to resolve the problem.
  2. HTTPRequest mutual independence.HTTPEach other is an independent individual request, in most cases when the client request web page is not a request can be successful, the server is the first responseHTMLPage, and then the browser receives the response and finds that the page references other resources, such as CSS, JS files, images, etc., and sends them automaticallyHTTPRequest these required resources.
  3. HTTPBased on the agreementTCPagreement.HTTPThe purpose of the protocol is to define the format of data transmission and data interaction between the client and the server. It is not responsible for the details of data transmissionTCPThe implementation. The current version is persistent by default, meaning multiple connectionsHTTPRequest to use aTCPThe connection.

Note: HTTP requests are not the same as TCP connections. HTTP is a transmission request initiated on the basis of TCP connections. Multiple HTTP requests can be sent under the same TCP connection channel, for example, the relationship between highways and cars.

HTTP history

The HTTP version 0.9

  • There is only oneGETCommand.
  • There are no request headers and no response headers to describe the data information associated with the transfer.
  • After sending data, the server shuts downTCPConnection, not supportedTCPPersistent connections.

The HTTP version 1.0

  • Added a lot of commands,HEAD,POST,PUT,DELETEAnd so on.
  • addedstatus codeStatus code andheaderRequest headers and response headers.
  • Added multi-character set support, multi-part sending, permissions, caching, and more.
  • Can be turned onConnection： keep-aliveTo specify the use ofTCPA long connection

HTTP 1.1 (currently in common use)

  • Persistent connections are supported by default
  • PersistentConnection is enabled by defaultConnection： keep-alive.
  • Support the Pipelining processing of requests, i.eTCPMultiple connections can be sentHTTPRequest and response.
  • increasedhostRequest header field, through thehostParsing allows multiple software services to run on the same physical server, greatly increasing server utilization. The currentnginxReverse proxy is based onHTTPRequest headerhostTo identify different requests and proxy them to different software services on the same server.

The HTTP 2.0

  • HTTP1.xIs based on text, there are analytic defects; whileHTTP2.0Use binary parsing insteadHTTP 1.XString parsing, more efficient and robust.
  • HTTP2.0All data is transmitted as “frames”, so multiple requests sent in the same connection no longer need to be returned in sequence and can be transmitted in parallel.
  • HTTP2.0Compress header information to optimize the amount of transmitted data.HTTP1.xThe request header carries a lot of information and is sent repeatedly each time,HTTP2.0useencoderTo reduce the size of the request headers that need to be transmitted, the communication parties cache a copyheader fieldsTable, not only avoids repeated transmission, but also reduces the size of the transmission information.
  • HTTP2.0A newserver push(server push) concept, the server can initiate some data push. For example, the server receives a message from the browserHTMLAt the same time, it can actively push relevant resource files (JS/CSS) to the client and send them in parallel to improve the transmission and rendering efficiency of web pages.
  • Currently if you want to useHTTP2Need to use firstHTTPSOnly on this basis can it be usedHTTP2

See the official demo of HTTP 2’s performance improvements for the most intuitive image loading performance improvements over HTTP 1

HTTPS

We often see floating pop-ups or ads on some web pages, and sometimes we even see these spam ads on our own live web pages. However, the developer has not written these things, but how does this kind of spam go up? The root cause lies in various proxy services. When we initiate an HTTP request from the client, it is not directly transmitted to the target server. During the process, we will go through layers of proxy services. Because HTTP uses plaintext strings to pass data, it can easily be read or tampered with by an intermediate service, which gets its hands on the raw HTML data and can easily insert a small AD into it.

HTTPS is a mechanism to solve the security problems arising from HTTP plaintext transmission ———— Encrypts the information in HTTP requests before transmission, effectively preventing the interception or tampering of information by intermediate proxy services.

HTTPS is simply HTTP 1.1, a more secure version, with a few caveats:

  1. HTTPSThe protocol needs to beCAApplication certificate, generally free certificate is very few, need to pay a fee
  2. HTTPThe protocol runs onTCPAbove, all transmitted content is clear text,HTTPSRunning on theSSL/TLSAbove,SSL/TLSRunning on theTCPAll transmissions are encrypted.
  3. HTTP 和 HTTPSThey’re using a completely different connection, and they’re using a different port. The former is80, which is443.
  4. HTTPSIt can effectively prevent operators from hijacking and solve a big problem of anti-hijacking.

HTTP packets

HTTP exists in the form of request and response. The initiator initiatively initiates an HTTP request, and the responder responds. The two parties exchange data according to a certain packet format.

The first line

The first line, which is not Http Headers, contains:

  1. HTTP methods (GET, POST, PUT, DELETE, etc.) have different meanings.

    HTTP Method Corresponding to
    GET It is used to obtain server resources
    POST Generally used to transport entity bodies
    PUT Generally used to transfer files
    DELETE Used to delete files
    HEAD It is used to obtain the packet head without returning the packet body
    OPTIONS Used in precheck requests to ask for methods supported by the request URI resource

    HTTP Method is just one of the specifications that the HTTP protocol espouses. Like ESLint, you can choose to follow or not follow, and they do essentially the same thing, just more semantically explicit.

  2. URL Specifies the address of the requested resource. This address will contain only the routing address of the request.

  3. The protocol version is HTTP 1.0 / HTTP 1.1 / HTTP 2.

  4. HTTP return status code (contained in the first line of the response message)

    HTTP defines 40 standard status codes that can be used to transmit the result of a client request. Status codes are divided into the following five categories. For information about returned status codes under each segment, refer to HTTP response codes:

One thing to note here is that a good HTTP application service should have a complete HTTP status code return message, i.e. the visitor can get the status information of the current HTTP request from the HTTP status > code alone. Most of our current development modes have only 200 and 500 HTTP return codes. The server will return the 200 and then tell you that there is something wrong with “not logged in”/” not authenticated “/” not authorized “. There is also a joke in the industry: it is not impossible to use, in fact, this kind of development method is not correct, regardless of the code maintenance and personal development, we need to > avoid this problem.

HTTP headers

HTTP headers. The information after the first line break is HTTP headers. HTTP headers generally contain non-business information about the interaction between the client and server, such as the data type of the request, request date, character set support, custom header fields, communication credentials, and cache support. HTTP Header Complete field list: portal

The main body

The HTTP body, HTTP Header and body information are distinguished by a blank line and a newline. The HTTP body typically places some business specific information for the request

HTTP Data negotiation

In THE HTTP protocol, data negotiation is a mechanism in which the client informs the server of the data format, presentation form, and data compression method through the request header. Common data negotiations include, for example, the natural language used in documents, the format of images, or the way content is encoded. The server can parse the data negotiation fields carried in the request header, and then inform the client of the data format and compression mode by using the relative fields when returning the data to the client. This allows the browser to parse, process, and render these resources in a specific way.

The following is a brief list of some commonly used data negotiation fields, complete data negotiation information portals

  • AcceptRequest header field that specifies the type of data to expect
  • Accept-EncodingRequest header field, specify the desired data to be obtained in what encoding mode of transmission, often used to limit the server side of the data compression method, common JS file package size optimization GZIP compression, used in this method
  • Accept-LanguageRequest header field, which specifies the expected data language: Chinese, English, or another language. This header field is usually added automatically by the browser
  • User-AgentThe request header field specifies the browser information of the request. The server can select a page with different compatibility based on this information and return it to the user or collect statistics on the browser information and operating system used by the user
  • Content-TypeResponse header field, in the request headerAcceptFields may specify several acceptable data formats, and the server will eventually return one data format to the client
  • Content-EncodingResponse header field, corresponding toAccept-Encoding
  • Content-LanguageResponse header field, corresponding toAccept-Language

Long HTTP connection

Every HTTP request needs to be sent and received over a TCP connection channel. In earlier versions of THE HTTP protocol, a new TCP connection channel was created before each HTTP request was sent, and when the request was completed, the TCP channel was automatically closed. The problem is that a single TCP connection cannot be reused, resulting in a lot of wasted new energy. Fortunately, this problem has been solved with the gradual improvement of HTTP protocol.

The Connection header field, introduced in HTTP 1.0, allows it to be keep-alive or Close to determine whether a TCP Connection needs to be reused or closed after a request is completed. In HTTP 1.1, this field is enabled on both ends by default, that is, HTTP long connections are supported by default.

Note the following: Connection: keep-alive The HTTP long Connection can be enabled only when both ends are enabled. If Connection is set to Close, the long Connection cannot be enabled because the establishment and persistence of TCP connections are a process of two ends interaction.

If you want to see TCP connection ID locally, you can open Chrome debugging tool to check:

In the figure, you can see that there are different Connection ids, which indicates that this request actually opens a new TCP Connection. The Connection ID of the lowest request is the same, which indicates that multiple HTTP requests reuse the same TCP Connection.

The maximum number of concurrent TCP connections supported by Chrome is six, and in HTTP versions below HTTP 2.0, requests are blocked. That is, once the six connections are full and the previous request is not completed, the subsequent request will be blocked until the previous request returns and the subsequent request can be sent.

HTTP cache

Although HTTP caching is not required, it is often necessary to reuse cached resources. However, common HTTP caches store only GET responses, not other types of responses. The key to caching is the Request Method and the target URI (usually only GET requests are cached).

Cache read strategy

The file cache in the front-end environment is divided into several different locations. When we open the Chrome console and look at the size option for each request record under Network, we will find a wealth of source information.

For the front-end browser environment, the cache read location is in order, the order is (search from top to bottom, find and return; If you can’t find it, continue)

  • Service Worker
  • Memory Cache
  • Disk Cache
  • Network request

Service Worker

The Service Worker’s cache differs from other built-in caching mechanisms in that it gives us control over which files are cached, how the cache is matched, and how the cache is read, and the cache is persistent.

  • Browser first.
  • Persistent storage.
  • You can control what is stored more flexibly, choose which files to cache, define routing rules for cached files, and so on.
  • You can view it from Chrome F12, Application -> Cache Storage.

Memory Cache

  • memory cacheIs cache storage in memory.
  • Fast reading speed.
  • Storage space is small.
  • Storage time is short when browsertabThe page is closed and the memory resource is freed.
  • If you specify it explicitlyCache-Control 为 no-storeBrowsers do notmemory-cache.

Disk Cache

  • Disk CacheIt’s cache storage on the hard disk.
  • The read speed is slower thanMemory CacheFaster than network requests.
  • Large storage space.
  • Persistent storage.
  • Disk CacheStrictly in accordance with theHTTPFields in the header to determine whether the resource is cacheable, authenticated, and so on.
  • Often heard are “mandatory caching”, “comparative caching”, as wellCache-ControlFall into this category.

Network request

If none of the resource files in a request match the above cache policy, the network request is initiated. When the browser gets the resource, it adds the new resource to the cache.

Cache-Control

The cache-control header defined in HTTP/1.1 is used to distinguish between support for caching mechanisms, and is supported by both request and response headers. Caching policies are defined by the different values it provides. It is important to note that cache-control is not suitable for scenarios where data changes rapidly.

instruction role
public Public cache: Indicates that the response can be cached by any middleman (such as intermediate proxy, CDN, etc.).
private Private cache: Indicates that the response is dedicated to a single user. The response cannot be cached by the middleman and can only be used in the browser’s private cache.
max-age (unit/second) Set the expiration time of the cache. After the expiration, a new request is required. Otherwise, the local cache is read and the request is not actually sent
s-maxage (unit/second) Overrides max-age. It has the same effect only on the proxy server
max-stale (in seconds) indicates that the expired cache is used even if the cache expires
no-store Disable caching
no-transform HTTP headers such as content-encoding, content-range, and Content-Type cannot be modified by the proxy. HTTP headers such as content-encoding, content-range, and Content-Type cannot be modified by the proxy. This directive is designed to prevent this).
no-cache Mandatory acknowledgement cache: Each time before using the local cache, request the server to check whether the cache is invalid. If the cache does not expire, the cache will use the local cache copy.
must-revalidate Cache validation validation: this means that when a cache is considered to use an obsolete resource, its state must be verified first. Expired caches will not be used
proxy-revalidate It has the same effect as must-revalidate, but applies only to shared caches (such as proxies) and is ignored by private caches.

Cache invalidation

To match the no-cache in cache-Control, we also need a mechanism to verify that the Cache is valid when the browser uses the Cache. For example, when the server resources are updated, the client needs to refresh the cache in time. Alternatively, the resource on the client expires, but the resource on the server is still old. In this case, resending the resource does not need to be done. Cache validation is used to address these issues, and in HTTP 1.1 we focused on last-Modified and ETag fields.

Last-Modified

As the name implies, is the time when the resource was last modified. When the client accesses the server resource, the server returns the last-Modified value to the client. When the client receives the last-Modified value, The next time the request is sent, the last-modified value returned by the server is loaded in if-modified-since or if-unmodified-since and sent to the server for cache verification.

This allows the server to read if-modified-since (more commonly) or if-unmodified-since values against local last-modified values. If the two values are the same, it means that the requested resource file has not been modified, and the server tells the browser that the resource is valid and can be used, otherwise it needs to use the latest resource.

Take a look at the two images below:

When you request the script.js script resource on the server, you can see that the server returns last-modified, which records the Last time the resource was Modified

When the client sends a request again, it carries the expiration time to the server for verification

Let’s look at some of the server code:

const http = require('http');
const fs = require('fs');
http.createServer((request, response) = > {
   const ifModifiedSince = request.headers['If-Modified-Since'];
   const lastModified = 'Web Aug 19 2019 19:01:15 GMT+0800 (China Standard Time)';
    
   if (request.url === '/') {
       const html = fs.readFileSync('test.html'.'utf-8');
       
       response.writeHead(200, {
           'Content-Type': 'text/html'
       });
       response.end(html);
   }
   if (request.url === '/script.js') {
       const js = fs.readFileSync('script.js'.'utf-8');
       let status = 200;
       // If the if-modified-since read is the same as lastModified, set the header 304 to indicate that the cache is available
       if (ifModifiedSince === lastModified) {
           status = 304;
           response.end(' ');
       }
       response.writeHead(status, {
           'Content-Type': 'text/javascript'.'Cache-Control': 'no-cache,max-age=2000'.'Last-Modified': lastModified }); response.end(js); }});Copy the code

ETag

The action of Etag is not fundamentally different from that of Last-Modified. Compared to last-Modified cache validation, which uses the Last Modified date to compare whether a resource is invalid, ETag uses data signatures to perform a more stringent cache validation.

The so-called data signature is actually a unique signature mark on the resource content. Once the resource content changes, the signature will be changed, and the server will use the signature as a secret to mark the validity of the cache. The typical approach is to perform a hash on the resource content, similar to the hash identifier added to the online resource by webPack

In the same way that last-modified corresponds to if-modified-since, ETag corresponds to if-match or if-none-match (if-none-match is more commonly used). If both signatures are the same, no new resource content is returned.

Proper use of cache validation

Last-modified and ETag only provide the server with a means to control the validity of the cache, but do not enforce the cache. The final decision whether to use the cache or use a new resource file depends on the server to specify the corresponding HTTP code. For files stored on the server, there is an attribute of the Last Modified date. When using last-Modified, this valid attribute can be used for data cache verification. Or store an UpdateTime field in the database to identify the specific modification date to determine whether the cache is valid. The details of how to build a server that can use caching properly involve more back-end knowledge, which is not described here.

The same origin policy of the browser

Same-origin restriction of the browser: If any of the schema/port/host used by the browser to access the URL does not match the current URL fragment information, the cross-domain problem will occur.

The current address Request the address Request successful
www.juejin.com:80 www.juejin.com:80 Cross-domain (different protocols)
www.juejin.com:80 www.juejin.cn:80 Cross domain (different domain names)
www.juejin.com:80 www.juejin.com:90 Cross-domain (different ports)

There are a few things to be clear about cross-domains:

  1. Cross-domain is a means of protection provided by the browserThere is no such thing as cross-domain server. Is that why the front end is more dependent on the front end in the current development model of front end separationwebpack-dev-serverThe reason for starting the proxy service to relay and the proxy backend interface is because there is no cross-domain barrier between the two servers to communicate with each other.
  2. Cross-domain, in the case of XMLHttpRequest, there are no cross-domain restrictions on the browser fetching static resources from different source servers, it is also aJSONPThe nature of cross-domain request implementation.
  3. Unlike XMLHttpRequest, when ascript resource is loaded through the SRC attribute, the browser limits the Javascript’s permissions to read, write, and return content.
  4. In addition to DOM, cookies, and XMLHttpRequest, common plug-ins such as Flash, Java applets, Silverlight, and Google Gears have their own control policies.

When a browser sends a request to a server in a different domain, the request actually goes out, the server actually receives the request, and it actually responds to your browser, and the browser actually receives valid data. However, in a cross-domain case, if the access-Control-Allow-Origin field in the header of the server’s return data is not whitelisted, the browser will hide the data from the server and not tell you. And throw you an Access-Control-Allow-Origin error.

Why are resource files not subject to the same origin policy? Imagine if resource files were restricted to cross domains, then the CDN caching strategy that is widely used today would be basically unusable. In addition, the resource files of many websites will be put on the OSS of cloud servers. The URL addresses of OSS resources must be in different domains, so these resources cannot be used.

Access-Control-Allow-Origin

Access-control-allow-origin specifies the cross-domain whitelist allowed by the server. It can be set in the following ways:

  1. Set up directly*Wildcard, simple and crude, but doing so is equal to the server all interface resources completely exposed, is not safe.
  2. Settings specify domains, such asAccess-Control-Allow-Origin: https://www.baidu.comThis only allows cross-domain access for requests from the specified domain.
  3. Dynamically set by the back end.Access-Control-Allow-OriginLimit writing to one whitelist, but what if we have multiple domains that require cross-domain requests? At this point, the server itself can maintain a set of whitelisted lists of the source of the request as it comes inhostWhitelisting, if it’s in the whitelist, thisAccess-Control-Allow-OriginSet it up dynamically and return the response.

CORS pre-request

If we set only access-Control-allow-Origin whitelist as above, will we be able to cross domains completely unimpeded? And it isn’t. Even if whitelisted domain name authentication is enabled on the peer end, some operations still need to be further authenticated. Such operations are CORS pre-requests.

The pre-request triggers the process

The triggering condition of the browser’s pre-request is to determine whether the request is a simple request. If the request is a complex one, the browser will prepare an HTTP Method named OPTIONS and send it as a pre-request before sending a formal cross-domain request. After the server has passed the pre-request, the following browsers do not make a formal data request. The entire request process is actually two requests: a precheck request and a subsequent request for actual data.

A simple request

  1. The request mode can only beGET POST HEAD
  2. The request header field only allows:
    • Accept
    • Accept-Language
    • Content-Language
    • Content-Type
  3. Content-TypeThe value of is limited to:
    • text/plain
    • multipart/form-data
    • application/x-www-form-urlencoded
  4. XMLHttpRequestUploadObjects are not registered with any event listeners (just be aware).
  5. Not used in the requestReadableStreamObjects (just know).

Complex request

All complex requests, except those defined in simple requests, require pre-requests.

Validation of pre-requests

So how can precheck requests be successfully authenticated? The server still needs to help set the request header whitelist:

  1. Access-Control-Allow-Headers, sets the additional request header fields allowed.
  2. Access-Control-Allow-Methods, sets the additional request methods allowed.
  3. Access-Control-Max-AgeIn units per second, specifies how long the results of a pre-request can be cached, within which further cross-domain requests are not pre-checked.

More specific cross-domain restriction policies can be viewed here

HTTP performance optimization solution

  1. The rational use ofHTTPTo avoid the extra performance overhead caused by multiple requests to the server for the same resource
  2. Try to useHTTPLong connections to avoid rebuilding each timeTCPTime loss associated with connection
  3. Try to useHTTPSTo ensure the security of network transmission.
  4. You can useHTTP2To greatly improve the efficiency of data transmission, usingserver pushopenHTTP2Server push function
  5. Client StartAccept-EncodingCompression mode support, the server to transfer compressed files, reduce the size of data transmission