This paper is participating in theNetwork protocols must be known and must be known”Essay campaign

You will Get skills

  • How to fool baidu server – with their own handwritten HTTP protocol

  • How to mess with Chrome – using your own handwritten HTTP protocol

  • Understand the relationship between HTTP and TCP

  • Understand the motivation behind the HTTP protocol

  • Compile an HTTP message word by word

Why

It is said that there is a very famous interview question in the front end called

What happens when you enter a URL

If the simple answer can be divided into

  • The DNS

  • A TCP connection

  • Sending an HTTP request

  • Server HTTP Reply

  • Browser parsing render

  • Connect the end of the

Of course, if you break down each stage in detail, it will be subdivided again. The so-called details determine success or failure. The more details you talk about, the more systematic your knowledge is. It’s up to you whether you can bluff or not, but this is a really good test.

Today, Uncle Ran only wants to talk about one of them.

How to send HTTP requests over TCP, and how to respond to HTTP requests.

The rules of the game

Practice is the sole criterion for testing truth.

So how do we verify that we can actually implement HTTP?

We know that HTTP protocol is divided into two parts: Request and Response

First we’ll use TCP to encapsulate HTTP. The success of the experiment was verified by the following two criteria.

  • Verification criteria for [Request] : Baidu replies can be received
  • Valid standard: can be accessed by Chrome browser

Where – Network base with TCP/IP

The first thing we need to do is figure out where we’re going to play the game, or it’s going to be a fairy fight.

We know that a network is essentially a network of at least two wires connecting nodes to exchange data.

Two sons, predictably, are still fighting, and the thousands of computers that keep them from fighting and getting along require complex protocols.

In the computer world the usual way to solve a complex problem is to solve it in layers

In fact, this is the OSI reference model, and what we have in the Internet today is the implementation of this theoretical model called TCP/IP protocol

What are TCP and HTTP

What is TCP communication?

In fact, there are two communication modes at the transport layer: TCP and UDP.

Both protocols can transmit data. The difference is whether to establish a connection in advance. TCP is one of the protocols that needs to establish a connection. So we say TCP does not lose packets.

However, UDP is not useless, such as playing games, a skill does not work, I press again, so a small delay is more important than reliable connection, so many early games are interested in UDP protocol.

For a high-level programming language (C++, Java, JS) generally can complete data transfer based on something called a socket.

TCP program

Let’s take a small example of Node.

Client

var net = require("net");
var client = net.connect(3000.() = > {
  console.log("Connect to the server!);
});

let n = 3;
const interval = setInterval(() = > {
  const msg = "Time " + new Date().getTime();
  console.log("Client sends:" + msg);
  client.write(msg);
  if (n-- === 0) {
    client.end();
    clearInterval(interval); }},500);

client.on("end".function () {
  console.log("Disconnect from the server");
});

Copy the code

Server

var net = require("net");
var server = net.createServer((connection) = > {
  console.log("client connected");
  connection.on("data".(data) = > {
    console.log("Server receives:" + data.toString());
  });
  connection.on("end".function () {
    console.log("Client closes connection");
  });
  connection.end("Hello I am \r\n");
});
server.listen(3000.function () {
  console.log("server is listening at 3000");
});

Copy the code

Why do you need HTTP

Now that we have seen above that we can send and receive data over TCP, suppose we want to do a request like BBS.

We can roughly compare BBS server to a library that stores texts, pictures, and even sounds and videos.

The user should fill out the loan form correctly if he/she wants to borrow or return a book. In this way, orderly access can be ensured.

Obviously, this function is not specified in THE TCP protocol. TCP only provides the possibility of exchanging data, which is equivalent to opening the small window of borrowing books. To actually complete the return of books, you need to design a check out list. In fact, this list is HTTP protocol.

HyperText Transfer Protocol (HTTP) is an application-layer Protocol for distributed, collaborative and hypermedia information systems. HTTP is the basis for data communication on the World Wide Web.

Let’s take a quick look at the HTTP protocol and see if it looks a lot like a borrowing order.

HTTP Protocol Rules

Let’s take a closer look at the HTTP protocol

To curl an HTTP packet, run the curl command

In fact, an HTTP packet is just text, which uses delimiters such as Spaces, carriage returns, and newlines to distinguish its parts.

Parsing HTTP Packets

Let’s start parsing an HTTP message in code.

The first step is to split the request line, header, and body

  • Request line: the first line – the character before the first carriage return and newline is the request line
  • Header: The line is requested until an empty line is encountered — essentially, two consecutive carriage returns and line feeds
  • Request body: the rest
 private parse(): void {
    this.httpMessage = {} as HttpMessage;
    const messages = this.message.split('\r\n');
    const [head] = messages;
    const headers = messages.slice(1, -2);
    const [body] = messages.slice(-1);
    this.parseHead(head);
    this.parseHeaders(headers);
    this.parseBody(body);
  }
Copy the code

The second step parses the request line

The request structure is: request method + space + URL + space + version number

private parseHead(headStr) {
    const [method, url, version] = headStr.split(' ');
    this.httpMessage.method = method;
    this.httpMessage.url = url;
    this.httpMessage.version = version;
  }
Copy the code

Step 3 Parse the header

The structure of the head:

KEY_A : VALUE

KEY_A : VALUE

KEY_C : VALUE

function parseHeaders(headers) {
    this.httpMessage.headers = {};
    for (let i = 0; i < headers.length; i++) {
      const header = headers[i];
      const [key, value] = header.split(":");
      key = key.toLocaleLowerCase();
      value = value.trim();
      this.httpMessage.headers[key] = value; }}Copy the code

Request body

The request body is the rest that doesn’t need parsing

Assemble the HTTP response

The assembly process is the reverse of the whole process

function format() {
  const head = `The ${this.version} The ${this.status} The ${this.message}`;
  let headers = ' ';
  for (let key in this.headers) {
  const value = this.headers[key];
  headers += `${key.toLocaleLowerCase()}: ${value}\r\n`;
  }
  const combineData = [head, headers, this.body].join('\r\n');
  return combineData;
}
Copy the code

HTTP crawler access baidu home page

The following uses the HTTP function that just wrote to assemble a message to tease [Baidu]

const net = require("net");

const createFormater = require("./http/formater");
const formater = createFormater("request");
const req = {
  method: "GET".url: "/".version: "HTTP / 1.1".headers: { "user-agent": "Curl / 7.71.1".accept: "* / *" },
  body: ""};console.log(formater.format(req))

const client = net.connect(80."www.baidu.com".() = > {
  console.log("Connect to the server!);
  client.write(formater.format(req));
});
client.on("data".function (data) {
  console.log(data.toString());
  client.end();
});
client.on("end".function () {
  console.log("Disconnect from the server");
});

Copy the code

We note that this program did not use HTTP protocol, but only sent a TCP request to Baidu, using the message is just my own implementation. Result Baidu server really answered. Flirtation success, that our HTTP protocol implementation is good.

Implement HTTP server that can be accessed by Chrome

Let’s see if this application stands up to Chrome’s test

const net = require("net");

const createFormater = require("./http/formater");
const formater = createFormater("response");
const res = {
  version: "HTTP / 1.1".status: "200".message: "OK".headers: {
    date: "Sat, 04 Dec 2021 14".connection: "keep-alive"."keep-alive": "timeout=5".// "content-length": "19",
  },
  body: "<h1> Hello HTTP<h1>"};const server = net.createServer(function (connection) {
  console.log("client connected");
  connection.on("data".(data) = > {
    console.log(data.toString());
  });
  connection.on("end".function () {
    console.log("Client closes connection");
  });
  connection.end(formater.format(res));
});
server.listen(3000.function () {
  console.log("server is listening");
});

Copy the code

The server sends the request and sends its own response to the browser, which renders the page correctly. The experiment can also be considered a success.

Summary review

At present, although the HTTP protocol is simply implemented, it is still very preliminary, and will be supplemented later

  • Picture, video and audio data
  • Cookie – session authentication
  • Cache implementation
  • The subcontract to upload
  • pipelines
  • Http2.0
  • Https and RSA certificates

We’ll make them happen

Wait for the follow-up update of The wheel made by Uncle Ran

Support but uncle

Pay attention to the full stack

Recent articles (thanks for your encouragement and support 🌹🌹🌹)

  • 🔥 🎄 Vue3 official tutorial 】 【 word notes | synchronous learning video 1050 praise
  • 🔥 39 exits run | Denver annual essay 100 praise
  • 🔥 Element3 Development Insider – Vue CLI plug-in development 167 likes
  • 🔥 make wheel series every day 500+ praise
  • 🔥 Vue3.0 global launch summary of dry goods 267 thumbs up

Welcome to clap brick, discuss more elegant implementation together