Did the picture trick you into getting in? Ha ha don't hurry to go, in fact, it is not related to the picture, we are just a break to continue those things. Sometimes when we write code, we upload/download images or files. If the connection is interrupted, the HTTP/FTP server or download software that does not have breakpoint continuation can only be retransmitted from the beginning. A good HTTP/FTP server or download software has breakpoint continuation capability, allowing users to continue transmission from the place where the upload/download connection is interrupted, which greatly reduces the user's annoyance.Copy the code

Scope request to get part of the content

In order to implement the interrupt recovery download requirement, we need to be able to download the specified download entity scope:

  • Range in the request header to specify the byte Range for the resource
  • The status code 206 response packet is returned
  • For multiple scope requests, the response will indicate multipart/byteranges in the header field content-Type. Let’s simulate this: First open our command line tool and write the following code (following is a picture I found on Baidu).
 curl -v --header "Range:bytes=0-10" http://pic26.nipic.com/20130117/10558908_124756651000_2.jpg
Copy the code

Run this to get the result:

From the information returned to us by the server in the HTTP section, we can see the following information (206 represents that the server agrees to our range request).

  • Content-type: image/ JPEG The content-type is an image
  • Accept-ranges: bytes The unit is bytes
  • Content-range: bytes 0-10/292871 The request segment ranges from 0 to 10, and the total number of bytes is 292871
  • Content-length: 11 The request Length is 11. With this in mind, we can simply simulate what the client and server side do:

Server-side code

First the client sends a header Range:bytes=0-10, then the server returns a header // accept-ranges :bytes // Content-range :0-10/ total size

// Get the range requestlet http = require('http');
let fs = require('fs');
let path = require('path');
let { promisify } = require('util'); // This module wraps something as a promise.let stat = promisify(fs.stat);

let server = http.createServer(async function (req, res) {

    let p = path.join(__dirname, 'con.txt'); // Determine the size of the current filelet statObj = await stat(p);
    let start = 0;
    let end = statObj.size - 1; // Read stream is packet before packet after packet:statObj.size is the size of the file.let total = end
    let range = req.headers['range'];
    if(range) {// Determine if the client has a request with range // tell it to support range request res.setheader ('Accept-Ranges'.'bytes'); / / /'Matched string'.'First group']
        letresult = range.match(/bytes=(\d*)-(\d*)/); start = result[1]? parseInt(result[1]):start; end = result[2]? parseInt(result[2])-1:end; // Content-range :0-10/ total size res.setheader ()'Content-Range', `${start}-${end}/${total}`)
    }
    res.setHeader('Content-Type'.'text/plain; charset=utf8');
    fs.createReadStream(p, { start, end }).pipe(res);
});
server.listen(3000);
Copy the code

So a simple server side is written.

Client code

// Write an options firstlet options = {
    hostname:'localhost',
    port:3000,
    path:'/',
    method:'GET'
}
let fs = require('fs');
let path = require('path');
let http = require('http');
let ws = fs.createWriteStream('./download.txt');
let pause = false;
letstart = 0; // Download 10 process.stdin.on('data'.function(chunk){
    chunk = chunk.toString();
    if(chunk.includes('p'){// Enter p to pause =true
    }else{
        pause = false; download(); }});function download(){
    options.headers = {
        Range:`bytes=${start}-${start+10}` } start+=10; // request http.get(options,function(res){
        let range = res.headers['content-range'];
        let total = range.split('/') [1];letbuffers = []; // Create a cache and store all read data in it. res.on('data'.function(chunk){
            buffers.push(chunk);
        });
        res.on('end'.function(){// Write the fetched data to a file. Write (buffer.concat (buffers));setTimeout(function() {if(pause === false&&start<total){
                    download();
                }   
            },1000)
        })
    })
}
download();
Copy the code

The code is very straightforward. Finally, we create a con.txt file in the same directory. Execute the client file with Node and it will do the trick.

Scene: the check

Sometimes, even if an endpoint initiates a continuation request when the URL corresponding to the request has changed on the server side, it is obvious that a unique way to identify the request is needed.

  • Etag(request header) ===> if-none-match
  • Last-modified ===> if-modified-since

Etag

Entity Tags (ETAGS) are designed to solve problems that Last-Modified could not solve.

  • Some files may change periodically, but the content does not change (only the modification time), so we do not want the client to think that the file has been modified and GET again.
  • Some files are Modified very frequently, for example, If they are Modified less than seconds (N times in 1s), and if-modified-since the granularity that can be checked is s-level, such changes cannot be determined (or the UNIX record MTIME is only accurate to seconds).
  • Some servers do not know exactly when a file was last modified.

To this end, HTTP/1.1 introduced Etag. An Etag is simply a tag associated with a file. It can be a version tag, for example: v1.0.0; Or 627-4D648041F6B80. The HTTP/1.1 standard does not specify what an Etag is or how it should be implemented. The only stipulation is that Etag should be placed in “”. Generally, the value of Etag is time + file size.

Last-Modified

If-modified-since, like last-modified, is an HTTP header used to record when a page was Last Modified, except that last-modified is an HTTP header sent from the server to the client. If-modified-since is the header sent by the client to the server. The client sends back the last-Modified timestamp from the server via the if-modified-since header. This allows the server to verify that the page is up to date. If it is not, it returns the new content. If the page is up to date, 304 is returned to tell the client that the page in its local cache is up to date, and the client can load the page directly from the local cache, thus greatly reducing the data transfer over the network, and also reducing the burden on the server. If you want to see how this works, look at caching in static services.

Are you itching to try?

Please give a thumbs-up if it helps you!