preface

Learning resources from Geek Time – Teacher Li Bing “Browser working principle and Practice”. Next, let’s check in every day

  • Day 01 Chrome architecture: Why 4 processes with only 1 page open?
  • Day 02 TCP: How to ensure that a page file can be delivered to the browser in its entirety?
  • Day 03 HTTP Request Flow: Why do many sites open quickly the second time?
  • Day 04 Navigation flow: What happens between entering the URL and presenting the page?

What will you learn from reading this article?

The navigation process is important. It is a bridge between the web loading process and the rendering process. If you understand the navigation process, you will be able to string together the entire page display process, which is the key to understanding how the browser works.

The main points of the article are as follows:

  1. The server can control the behavior of the browser based on the response header, such as jump, network data type determination.
  2. Chrome defaults to one render process for each TAB, but if two pages belong to the same site, both tabs will use the same render process.
  3. The browser navigation process covers all the intermediate stages from the user initiating the request to submitting the document to the rendering process.

Display the complete process diagram from entering URL to page:

As you can see, the process requires coordination between processes, so before we start the formal process, let’s quickly review the main responsibilities of the browser process, renderer process, and web process.

  • Browser processes are responsible for user interaction, child process management, and file storage.
  • Web processes are web downloads for renderers and browser processes.
  • The rendering process is mainly responsible for parsing HTML, JavaScript, CSS, images and other resources downloaded from the Web into pages that can be displayed and interacted with. Because all the contents of the renderer process are obtained through the network, there will be some malicious code to exploit browser vulnerabilities to attack the system, so the code running in the renderer process is not trusted. This is why Chrome makes the rendering process run in a security sandbox, just to keep the system safe.

After reviewing the browser process architecture, let’s look at the complete process in the figure above. As you can see, there are many steps involved in the process, and I’ve highlighted some of the core nodes on a blue background. The process can be roughly described as follows.

  • First, the browser process receives the URL request entered by the user, and the browser process forwards the URL to the network process.
  • The actual URL request is then made in the network process.
  • The network process then receives the response header data, parses it, and forwards it to the browser process.
  • After receiving the response header data from the network process, the browser process sends a “CommitNavigation” message to the renderer process.
  • After receiving the “submit navigation” message, the renderer process is ready to receive THE HTML data by directly establishing a data pipeline with the network process.
  • Finally, the renderer process “confirms submission” to the browser process, which tells the browser process that it is ready to accept and parse the page data.
  • When the browser process receives a “submit document” message from the renderer, it removes the old document and updates the page state in the browser process.

Among them, the user sends a URL request to the page to start the process of parsing, called navigation.

From entering URL to page display

1. User input

When a user enters a query keyword in the address bar, the address bar determines whether the entered keyword is the search content or the requested URL.

  • For search content, the address bar uses the browser’s default search engine to synthesize new urls with search keywords.
  • If the input content complies with THE URL rules, for example, time.geekbang.org, the address bar combines the content with the protocol according to the rules to create a complete URL, such as time.geekbang.org.

When the user enters the keyword and enters Enter, it means that the current page is about to be replaced with a new page, but before the process continues, the browser also gives the current page the opportunity to perform a beforeUnload event, which allows the page to perform some data cleaning before exiting. Users can also be asked if they want to leave the current page, for example if the current page may have unfinished forms, so users can unnavigate by using the beforeUnload event without the browser doing any subsequent work.

The current page does not listen to the beforeUnload event or agrees to continue the process, so the browser enters the state shown below:

As you can see from the figure, the icon on the TAB page enters the loading state as soon as the browser starts loading an address. At this point, however, the page in the image still displays the content of the previously opened page and is not immediately replaced with the Geek time page. The content of the page will not be replaced until the submission stage.

2. URL request process

Next, you enter the page resource request process. In this case, the browser process sends the URL request to the network process through interprocess communication (IPC). After receiving the URL request, the network process initiates the actual URL request process here. So what’s the process?

First, the network process looks up whether the local cache has cached the resource. If there is a cached resource, it is returned directly to the browser process. If the resource is not found in the cache, the network request flows directly. The first step before the request is to perform a DNS resolution to get the server IP address for the requested domain name. If the request protocol is HTTPS, you also need to establish a TLS connection.

The next step is to establish a TCP connection with the server using the IP address. After the connection is established, the browser side will construct the request line, request information, etc., and attach the data related to the domain name, such as cookies, to the request header, and then send the constructed request information to the server.

After receiving the request information, the server generates response data (including response line, response header, and response body) based on the request information and sends it to the network process. After the network process receives the response line and header, it parses the contents of the header. (For the sake of illustration, I refer to the response headers and response rows returned by the server as response headers below.)

(1) Redirect

Upon receiving the response header from the server, the network process begins to parse the response header. If the status code returned is 301 or 302, the server needs the browser to redirect to another URL. The network process reads the redirected address from the Location field in the response header, then initiates a new HTTP or HTTPS request and starts all over again.

For example, we enter the following command in the terminal:

curl -I http://time.geekbang.org/
Copy the code

The curl -i + URL command receives the information in the response header returned by the server. After executing the command, we see the following response header returned from the server:

As you can see from the figure, the Geek time server converts all HTTP requests into HTTPS requests through redirection. This means that when you make an HTTP request to the Geektime server, the server will return a response header containing either a 301 or 302 status code and fill in the Location field of the response header with an HTTPS address, which tells the browser to navigate to the new address.

Let’s make a geek time request using HTTPS to see what the server’s response header looks like.

curl -I https://time.geekbang.org/
Copy the code

We see the following message from the server:

As you can see from the figure, the server returns a response header with a status code of 200, which tells the browser that everything is fine and that it is time to proceed with the request.

Ok, so that’s the redirection. Now you should understand that during navigation, if the status code in the response line of the server contains a jump of 301 or 302, the browser will go to the new address to continue navigation. If the response line is 200, then the browser can continue processing the request.

(2) Response data type processing

After processing the jump information, we continue the analysis of the navigation flow. The data type of the URL request, sometimes a download type, sometimes a normal HTML page, so how do browsers distinguish between them?

The answer is content-type. The Content-Type is a very important field in the HTTP header that tells the browser what Type of response body data the server is returning. The browser then uses the value of the Content-Type to decide how to display the response body Content.

Let’s take geek Time as an example and see what the Content-Type value is returned by Geek Time. Enter the following command on the terminal:

curl -I https://time.geekbang.org/
Copy the code

The following information is displayed:

As you can see, the content-Type field in the response header is text/ HTML, which tells the browser that the data returned by the server isHTML format.

Curl curl curl curl curl curl curl curl curl curl curl

curl -I https:/ / res001.geekbang.org/apps/geektime/android/2.3.1/official/geektime_2.3.1_20190527-2136_offical.apk
Copy the code

The following response header is returned after the request:

The content-type of the response header is application/octet-streamByte stream typeNormally, the browser will followThe download typeTo process the request.

Note that if the content-type is incorrectly configured on the server, such as setting the text/ HTML Type to application/octet-stream, the browser may misinterpret the file’s Content, for example, by turning a page intended for presentation, It becomes a download file.

Therefore, the subsequent processing flow of different Content-Types is quite different. If the value of the Content-Type field is determined by the browser to be a download Type, the request is submitted to the browser’s download manager and the navigation of the URL request ends. But if it’s HTML, the browser will continue with the navigation process. Since Chrome’s page rendering runs in the render process, the next step is to prepare the render process.

3. Prepare the rendering process

By default, Chrome assigns a render process to each page, meaning that a new render process is created for each new page opened. However, there are some exceptions, in some cases the browser will allow multiple pages to run directly in the same render process.

For example, I opened up another page from geek Time’s home page, Algorithmic Boot Camp. Let’s take a look at Chrome’s task Manager screenshot below:

As you can see from the figure, the three open pages are all running in the same render process with the process ID 23601.

When can multiple pages be running in a render process at the same time?

To solve this problem, we need to understand what same-site is. Specifically, we define “same site” as the root domain (for example, geekbang.org) plus the protocol (for example, https:// or http://), plus all the subdomains and different ports under the root domain, such as the following three:

https://time.geekbang.org
https://www.geekbang.org
https://www.geekbang.org:8080
Copy the code

They all belong to the same site because their protocol is HTTPS and the root domain name is geekbang.org.

Chrome’s default strategy is one render process per TAB. However, if a new page is opened from one page and belongs to the same site as the current page, the new page will reuse the parent page’s rendering process. Officially, this default policy is called process-per-site-instance.

What happens when the new page and the current page are not on the same site? For example, I went to InfoQ’s website via a link on geekbang’s page (www.infoq.cn/), infoq.cn and Geekbang.org do not belong to the same site, so infoq.cn will use a new rendering process, as you can see below:As can be seen from the task manager in the figure, the tabs owned by Geek Bang and Geek TimeSame protocol and root domain, soThey belong to the same siteAnd run in the same render process; Infoq.cn has a different root domain from Geekbang.org, meaning that Infoq and Geekbang are not part of the same site, so they run in two different rendering processes.

To summarize, the rendering process strategy used to open a new page is:

  • Typically, a separate rendering process is used to open a new page;
  • If page B is opened from page A, and A and B belong to the same site, then page B reuses the rendering process of page A; If otherwise, the browser process creates a new renderer for B.

Once the renderer process is ready, it cannot immediately enter the document parsing state because the document data is still in the network process and has not been submitted to the renderer process, so the next step is to submit the document.

4. Submit documents

Submitting a document means that the browser process submits the HTML data received by the web process to the renderer process. The process looks like this:

  • First, when the browser process receives the response header data from the network process, it sends a “submit document” message to the renderer process.
  • After receiving the “submit document” message, the renderer process establishes a “pipeline” with the network process to transfer data.
  • After the document data transfer is complete, the renderer process returns a “confirm submit” message to the browser process.
  • After receiving the “confirm submission” message, the browser process updates the browser interface status, including the security status, the URL of the address bar, the historical status of forward and backward, and the Web page.

Where, when the rendering process confirms the submission, the update content is as shown below:

This explains why, when you type an address into your browser’s address bar, the previous page doesn’t disappear immediately, but instead takes a while to load before the page is updated.

At this point, a complete navigation flow is “gone”, after which it is time to enter the rendering phase.

5. Rendering phase

Once the document is submitted, the rendering process begins page parsing and subresource loading, which I’ll cover in the next article. The only thing you need to know here is that once the page is generated, the renderer will send a message to the browser process, which will stop the loading animation on the TAB icon when it receives the message. As follows:

At this point, a complete page is generated. The “What happens between entering the URL and presenting the page?” question at the beginning of the article. This process and its “cascade” problem are solved.

Answering questions

keep-alive

First of all, keep-alive is to solve the problem of low connection efficiency. In http1.0 era, HTTP requests are in the form of short connection, that is, each request for a resource needs to establish a connection with the server, transmit data, and disconnect. Usually, the time for establishing a connection and disconnecting may exceed the time for transmitting data. The efficiency of this short connection is unusually inefficient.

Keep-alive (keep-alive) : Keep-alive (keep-alive)

You can take a long connection as a pipeline, an HTTP request is done, will not close the connection, the next request can reuse the connection, it connect and disconnect the time, but they request is in accordance with the order, which is adaptable to the rules of the IP + port resources can reuse the connection, this answer the above mention this problem.

However, using keep alive – the same problems, such as a page might be 100 images material, assuming that these pictures below material is stored in the same domain name, if only reuse a HTTP pipeline, then transmit 100 pictures of material is also very time consuming, it appeared the same time the demand of the concurrent connections server, As mentioned in the article, only 6 requests can be made under the same domain name at the same time, which can greatly improve the request efficiency.

Why 6 requests instead of more? It is for the sake of server performance. If there are unlimited connections at the same time, the server may be overwhelmed.