preface

When we upload a file, if the file content is small, we can directly convert the file to byte stream and transfer it to the server. However, if you encounter large files, this method is very painful to the user, and in case of mid-transmission interruption, you have to start from zero again.

Therefore, we need to adopt fragment upload/breakpoint continue to optimize the user experience.

Breakpoint continuingly

Breakpoint continuation is to skip the last uploaded file content, slice the file content using blob. slice method, and directly upload the remaining bytes.

To monitor the progress

If we want to implement breakpoint continuation, we need to know how much progress we have uploaded before. We use XMLHttpRequest for the upload because fetch can’t listen for progress events (or maybe it’s my fault).

We use xhr.upload. Onprogress to implement progress monitoring. To implement breakpoint continuation, we need to know the number of bytes received by the server. So in addition to the upload request, we need to add a request asking how many bytes the server uploaded.

Implementation approach

First we create a unique id to identify the file we want to upload.

let fileId = file.name + The '-' + file.size + The '-' + file.lastModified;
Copy the code

If the file name, size, and last modified date have changed, the breakpoint will determine that the file is a new file and generate a new fileId. That means there will be no sequel.

Sends a request to the server asking how many bytes the server has received.

let response = await fetch('status', {
  headers: {
    'X-File-Id': fileId
  }
});

// The number of bytes on the server
let startByte = +await response.text();
Copy the code

The server processes the current File by getting the x-file-ID header to get the fileId we just set. When there is no File on the server, the response should be 0.

We then use the blob. slice method to send the file after startByte.

xhr.open("POST"."upload".true);

// File unique identifier
xhr.setRequestHeader('X-File-Id', fileId);

// The number of bytes to start uploading
xhr.setRequestHeader('X-Start-Byte', startByte);

xhr.upload.onprogress = (e) = > {
  console.log(`Uploaded ${startByte + e.loaded} of ${startByte + e.total}`);
};


xhr.send(file.slice(startByte));
Copy the code

Here, we send fileId as x-file-ID to the server so it knows what File we’re uploading, and startByte as x-start-byte tells the server if we’re uploading initially or continuing, It depends on whether startByte is 0.

If the server finds that startByte is not 0, it should append the byte stream to the field file.

The core code here references the GiHub code.

index.html

<! DOCTYPEHTML>

<script src="uploader.js"></script>

<form name="upload" method="POST" enctype="multipart/form-data" action="/upload">
  <input type="file" name="myfile">
  <input type="submit" name="submit" value="Upload (Resumes automatically)">
</form>

<button onclick="uploader.stop()">Stop upload</button>


<div id="log">Progress indication</div>

<script>
  function log(html) {
    document.getElementById('log').innerHTML = html;
    console.log(html);
  }

  function onProgress(loaded, total) {
    log("progress " + loaded + '/' + total);
  }

  let uploader;

  document.forms.upload.onsubmit = async function(e) {
    e.preventDefault();

    let file = this.elements.myfile.files[0];
    if(! file)return;

    uploader = new Uploader({file, onProgress});

    try {
      let uploaded = await uploader.upload();

      if (uploaded) {
        log('success');
      } else {
        log('stopped'); }}catch(err) {
      console.error(err);
      log('error'); }};</script>
Copy the code

uploader.js

class Uploader {

  constructor({file, onProgress}) {
    this.file = file;
    this.onProgress = onProgress;

    // create fileId that uniquely identifies the file
    // we could also add user session identifier (if had one), to make it even more unique
    this.fileId = file.name + The '-' + file.size + The '-' + file.lastModified;
  }

  async getUploadedBytes() {
    let response = await fetch('status', {
      headers: {
        'X-File-Id': this.fileId
      }
    });

    if(response.status ! =200) {
      throw new Error("Can't get uploaded bytes: " + response.statusText);
    }

    let text = await response.text();

    return +text;
  }

  async upload() {
    this.startByte = await this.getUploadedBytes();

    let xhr = this.xhr = new XMLHttpRequest();
    xhr.open("POST"."upload".true);

    // send file id, so that the server knows which file to resume
    xhr.setRequestHeader('X-File-Id'.this.fileId);
    // send the byte we're resuming from, so the server knows we're resuming
    xhr.setRequestHeader('X-Start-Byte'.this.startByte);

    xhr.upload.onprogress = (e) = > {
      this.onProgress(this.startByte + e.loaded, this.startByte + e.total);
    };

    console.log("send the file, starting from".this.startByte);
    xhr.send(this.file.slice(this.startByte));

    // return
    // true if upload was successful,
    // false if aborted
    // throw in case of an error
    return await new Promise((resolve, reject) = > {

      xhr.onload = xhr.onerror = () = > {
        console.log("upload end status:" + xhr.status + " text:" + xhr.statusText);

        if (xhr.status == 200) {
          resolve(true);
        } else {
          reject(new Error("Upload failed: "+ xhr.statusText)); }};// onabort triggers only when xhr.abort() is called
      xhr.onabort = () = > resolve(false);

    });

  }

  stop() {
    if (this.xhr) {
      this.xhr.abort(); }}}Copy the code

conclusion

In fact, there are still some disadvantages of this method when transferring large files. For example, when the server has upload size limit, it will still fail to upload.

Since the blob. slice method is used, you can simply slice the file. Add a regular header for each file block and upload it separately.

After all the blocks are sent, a merge request is sent to allow the server to concatenate the blocks. Here also sorted out a few more useful upload components to share:

  • resumable-file-uploads
  • vue simple uploader
  • web-uploader
  • Resumable.js

Feel the arrangement of a good point of praise 😂.