Write a paragraph of nonsense to warm up

Although man-in-the-middle attacks are mentioned, this is not a security article, and to modify HTTPS content through a man-in-the-middle, the client must trust the certificate provided by the man-in-the-middle.

I do such a job, the most original requirement is to solve the problem of NPM package installation in the internal network environment of the company. To put it simply, it is to switch the warehouse and rely on the mirror source. The commonly used CNPM also provides mirroring capabilities and resolves the problem of hard-coded addresses that depend on packages, but does not support lockfiles or packages defined in the manner of URLs as Dependencies. Finally, we decided to use proxy to respond to the request from the Internet with Intranet resources.

The whole process, really fully feel the difficulty of modifying HTTPS request, after all, HTTPS was born to prevent content theft, tampering.

The proxy implementation of HTTP requests is not so complicated, so I will skip it for now.

Results demonstrate

I started a proxy server locally and injected some configuration to redirect access to www.google.com.hk to an HTTPS server that I was running on my machine.

The demonstration here, with the WAY of URL replacement, this part belongs to the specific business logic, after the final implementation of the simplified version, although the effect is the same ~

Background knowledge

Before implementing the proxy service, you can take a brief look at the HTTPS service certificate authentication process and how the proxy works.

Certificate: CA certificate and domain name certificate

To set up a normal HTTPS server, we need to apply for a domain name certificate from a certificate authority, where the authority must be trusted. The credential process is based on the trust chain. If the computer trusts the CA, it also trusts the domain name certificate issued by the CA certificate.

So there are two certifications involved:

  1. Institutional certification, the corresponding CA certificate, is preset in the system
  2. Domain name certificate, using the CA certificate to sign the domain name certificate, get a domain name certificate

You can create your own CA certificate to sign various domain names, also known as a self-signed certificate. Self-signed certificates are not verifiable and you need to get the client to trust your CA.

Proxy and direct access

What is the difference between a proxy for HTTP (S) and a normal request? How does the client tell the proxy the address of the target server? I took an image from someone else’s article:

Proxy-connection in the Http request header

Function implementation

A simple tunnel broker

picture

Word version

  1. Set up an HTTPS server as a proxy server
  2. Listen for connect events to get target server address, port, and ClientSocket
  3. Establish a connection with the target server, get the TargetSocket, and notify the client that the connection was established successfully
  4. Forward ClientSocket and TargetSocket data streams to each other

Code version

/** ** */

const https = require('https');
const fs = require('fs');
const forge = require('node-forge');
const net = require('net');

function connect(clientRequest, clientSocket, head) {
    constprotocol = clientRequest.connection? .encrypted ?'https:' : 'http:';
    const { port = 443, hostname } = url.parse(`${protocol}//${clientRequest.url}`);

    // Connect to the target server
    const targetSocket = net.connect(port, targetUrl, () => {
        // Notify the client that a connection has been established
        clientSocket.write(
            'the HTTP / 1.1 200 Connection Established \ r \ n'
                + 'Proxy-agent: MITM-proxy\r\n'
                + '\r\n',);// Set up a communication tunnel to forward data
        targetSocket.write(head);
        clientSocket.pipe(targetSocket).pipe(clientSocket);
    });
}

// Create a domain name certificate and start the HTTPS service as the proxy server
const serverCrt = createServerCertificate('localhost');
https.createServer({
        key: forge.pki.privateKeyToPem(serverCrt.key),
        cert: forge.pki.certificateToPem(serverCrt.cert),
    })
    .on('connection'.console.log)
    .on('connect', connect) // Set up a communication tunnel
    .listen(6666, () = > {console.log('Proxy server started, proxy address: https://localhost:6666');
    });
Copy the code

Question: What is the certificate authentication process in HTTPS proxy mode?

The code implementation above may not seem very nutritious, but it helps to understand the certificate authentication process in proxy mode. The above process involves two certifications:

  1. The proxy server is the HTTPS service. The connection between the client and the proxy server requires authentication
  2. The client needs to verify the certificate of the target server and generate encrypted session data for subsequent communication

Well, the question is,

  1. When did the two certifications take place?
  2. When are connection and connect events triggered in the code?

Try setting the proxy address to https://127.0.0.1:6666 (note that the proxy domain name is changed and the proxy service certificate verification is problematic) then you will see the connection event fired and tell you that the client has voluntarily disconnected… And then… There is no next.

If the certificate of the proxy server is authenticated, the Connection and connect events are triggered.

The certificate of the target server that the client needs to verify is passed to the client through PIPE after the proxy service establishes a connection with the target server.

Question: What happens if the proxy server establishes a connection not to the target server but to another server?

The second of the two authentications, the client’s certificate to the target server, failed and the connection was disconnected.

So, if we ask “the other server” to respond with the correct certificate, or “fake the target server”, can we establish the connection correctly, and then… Got your way?

Forge the target server

Certificate question: How to provide a certificate for any domain name?

When we set up an HTTPS server, we usually need to apply for a domain name certificate and find a CA that the client trusts to sign for you.

Therefore, in fact, let the certificate available conditions is relatively simple, with the client trusted CA certificate to sign a domain name certificate, it is ok.

In my target scenario, THE client is under my own control, so it is feasible to create a CA certificate to be trusted by the client. Since the CA is trusted, the domain certificate is signed arbitrarily.

Fake an HTTPS service to handle requests from multiple domain names

Since the destination address of the proxy is uncertain, it could be a.com or B.cn, I expect to build an HTTPS service to handle requests from different domain names.

There is something called “SNI,” or “server name indication,” for implementing one-to-many relationships between services and domain names.

/** ** */

/** Create an HTTPS service that supports multiple domains **/
function createFakeHttpsServer() {
    return new https.Server({
        SNICallback: (hostname, callback) = > {
            const { key, cert } = createServerCertificate(hostname);
            callback(
                null,
                tls.createSecureContext({
                    key: forge.pki.privateKeyToPem(key),
                    cert: forge.pki.certificateToPem(cert), }), ); }}); }const fakeServer = createFakeHttpsServer();

/** here is the concrete business, return to the client to provide the content **/
fakeServer.on('request', (req, res) => {
    // do something
    // At this point, the certificate part has passed, and the normal response to the request is ok
    res.writeHead(200);
    res.end('hello world\n');
}).listen(0);
Copy the code

Replace HTTPS site content with a proxy server

To summarize the steps above:

  1. Create a fakeServer fakeServer
  2. Example create proxyServer proxyServer
  3. ProxyServer listens for connection requests from clients
  4. ProxyServer establishes a connection to fakeServer
  5. ProxyServer establishes a communication tunnel between client requests and fakeServer
  6. FakeServer handles client requests based on business needs
/** createServerCertificate ()

const https = require('https');
const fs = require('fs');
const forge = require('node-forge');
const net = require('net');
const tls = require('tls');
const url = require('url');
const createServerCertificate = require('./cert');

function connect(clientRequest, clientSocket, head) {
    // Connect to the target server
    const targetSocket = net.connect(this.fakeServerPort, '127.0.0.1', () = > {// Notify the client that a connection has been established
        clientSocket.write(
            'the HTTP / 1.1 200 Connection Established \ r \ n'
                + 'Proxy-agent: MITM-proxy\r\n'
                + '\r\n',);// Set up a communication tunnel to forward data
        targetSocket.write(head);
        clientSocket.pipe(targetSocket).pipe(clientSocket);
    });
}

/** Create an HTTPS service that supports multiple domains **/
function createFakeHttpsServer(fakeServerPort = 0) {
    return new Promise((resolve, reject) = > {
        const fakeServer = new https.Server({
            SNICallback: (hostname, callback) = > {
                const { key, cert } = createServerCertificate(hostname);
                callback(
                    null,
                    tls.createSecureContext({
                        key: forge.pki.privateKeyToPem(key),
                        cert: forge.pki.certificateToPem(cert),
                    }),
                );
            },
        })
        fakeServer
            .on('error', reject)
            .listen(fakeServerPort, () => {
                resolve(fakeServer);
            });
    });
}

function createProxyServer(proxyPort) {
    return new Promise((resolve, reject) = > {
        const serverCrt = createServerCertificate('localhost');
        const proxyServer = https.createServer({
            key: forge.pki.privateKeyToPem(serverCrt.key),
            cert: forge.pki.certificateToPem(serverCrt.cert),
        })
        .on('error', reject)
        .listen(proxyPort, () => {
            const proxyUrl = `https://localhost:${proxyPort}`;
            console.log('Proxy started successfully, proxy address:', proxyUrl);
            resolve(proxyServer);
        });
    });
}

// Business logic
function requestHandle(req, res) {
    res.writeHead(200);
    res.end('hello world\n');
}

// This is the entrance
function main(proxyPort) {
    return Promise.all([
        createProxyServer(proxyPort),
        createFakeHttpsServer(), // Random port
    ]).then(([proxyServer, fakeServer]) = > {
        // Establish a communication tunnel between the client and the pseudo server
        proxyServer.on('connect', connect.bind({
            fakeServerPort: fakeServer.address().port,
        }));
        // Pseudo server processing, can respond to custom content
        fakeServer.on('request', requestHandle);
    }).then((a)= > {
        console.log('everything is ok');
    });
}

// Listen for exceptions to avoid an unexpected exit
process.on('uncaughtException', (err) => {
    console.error(err);
});

main(6666);
Copy the code

Complete code attached

The source address

Run the demo

  1. Starting the proxy Server
cd demo/proxy
npm i
npm run test
Copy the code
  1. Will the demo/proxy/cert/cacert. Pem import system and trust
  2. Set the browser proxy to http://localhost:6666
  3. Access any HTTPS site

No, please delete the certificate ~~

Reference documentation

  1. Why IS HTTPS Secure
  2. Proxy-connection in the Http request header
  3. Nodejs document – the TLS
  4. Principle and Implementation of HTTP Proxy (PART 1)
  5. Principle and Implementation of HTTP Proxy (II)
  6. Creating a CA Certificate