In this paper, We will introduce how the front end performs image processing according to the following process, and then introduce the knowledge points related to binary, Blob, Blob URL, Base64, Data URL, ArrayBuffer, TypedArray, DataView and image compression.


After reading this article, you should be able to easily understand the following transformation diagram:


What are you hesitating about? Following in the footsteps of Bob, let’s play with front-end binary. Please forgive me for my narcissism. In the following examples, we will use my personal avatar as the demo material.

Read Po’s recent popular articles (thanks to Digg friends for their encouragement and support 🌹🌹🌹) :

  • 1.2 W word | great TypeScript introductory tutorial (1259 + πŸ‘)
  • Top 10 Cool hover effects (686+ πŸ‘)
  • Top 10 AMAZING TS Projects (679+ πŸ‘)
  • Understanding TypeScript generics and applications (7.8k words) (549+ πŸ‘)
  • Image processing need not worry, give you ten small helper (454+ πŸ‘)

Ok, now let’s go to the first step: “Select local image -> Image Preview”.

Select Local Image -> Image Preview

1.1 FileReader API

In browsers that support the FileReader API, you can also make it easy to preview images locally.


(Photo credit: https://caniuse.com/#search=filereader)

As can be seen from the above figure, the API is compatible and we can use it with confidence. Without going into the details of the FileReader API, let’s take a look at how to implement a local image preview using the FileReader API:


      
<html>
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="Width = device - width, initial - scale = 1.0" />
 <title>Image local preview example</title>  </head>  <body>  <h3>Po: Image local preview example</h3>  <input type="file" accept="image/*" onchange="loadFile(event)" />  <img id="previewContainer" />   <script>  const loadFile = function (event) {  const reader = new FileReader();  reader.onload = function () {  const output = document.querySelector("#previewContainer");  output.src = reader.result;  };  reader.readAsDataURL(event.target.files[0]);  };  </script>  </body> </html> Copy the code

In the above example, we bind the onchange event handler loadFile to the input box of type file. In this function, we create a FileReader object and bind the corresponding onload event handler to the object. Then call the readAsDataURL() method of the FileReader object to convert the File object corresponding to the local image into a DataURL.

❝

In addition to converting File/Blob objects to Data urls, FileReader also provides readAsArrayBuffer() and readAsText() methods. Used to convert File/Blob objects to other data formats.

❞

When the file is finished reading, the bound onLoad event handler is triggered. Inside the handler, the acquired Data URL is assigned to the SRC attribute of the IMG element, thereby implementing a local preview of the image.

Using Chrome developer tools, we can see the “face” of the Data URL in the Elements panel:


In the green box on the right, we can clearly see that the img element SRC attribute value is a very “strange” string:

data:image/png; base64,iVBORw0KGgoAAAANSUhEUgAAAhAAAAIwCAYAAADXrFK...Copy the code

In fact, this strange string is called a Data URL, and it consists of four parts: a prefix (Data:), a MIME type indicating the Data type, an optional Base64 tag if it’s not text, and the Data itself:

data:[<mediatype>][;base64],<data>
Copy the code

Mediatype is a string of MIME type, such as “image/ PNG “for a PNG image file. If omitted, the default is text/plain; Charset = US – ASCII. If the data is of text type, you can embed the text directly (using the appropriate entity character or escape character, depending on the document type). For binary data, you can base64 encode the data and then embed it.

❝

MIME (Multipurpose Internet Mail Extensions) is an application type for opening a file with a specified extension name. When the file is accessed, the browser automatically opens the file with a specified application name. It is used to specify the name of a file customized by the client and the opening mode of some media files.

Common MIME types are: hypertext Markup language text. HTML text/ HTML, PNG image. PNG image/ PNG, and plain text.

❞

In order to reduce the number of HTTP requests during The development of Web projects, we often consider using Data URLS embedded in HTML or CSS files for smaller ICONS. “However, it is important to note that if the image is large and has a rich color hierarchy, it is not suitable to use this method because the base64 encoded string of the image is very large, which can significantly increase the size of the HTML page and affect the loading speed.”

In the Data URL, the Data is an important part, which is represented using base64 encoded strings. So to understand Data urls, we also need to understand Base64.

1.2 Base64

“Base64” is a representation of binary data based on 64 printable characters. Since “2⁢ = 64”, each 6 bits is a cell corresponding to a printable character. Three bytes have 24 bits, corresponding to four base64 units, that is, three bytes can be represented by four printable characters. The corresponding conversion process is shown in the figure below:


“Base64 is used to represent, transmit, and store binary data, including MIME E-mail and COMPLEX XML data, in text-based applications.” In MIME format E-mail, Base64 can be used to encode binary byte sequence data into text composed of ASCII character sequences. When used, specify Base64 in transport encoding mode. The characters include 26 uppercase and lowercase Latin letters, 10 digits, plus sign (+), and slash (/). The equals sign (=) is used as suffixes.

The corresponding indexes of Base64 are as follows:


After knowing the above knowledge, we take coding Man as an example to intuitively feel the coding process. Man consists of M, A, and N. The corresponding ASCII codes are 77, 97, and 110.


Then we carry out base64 encoding operation in every 6 bits as a unit, as shown in the figure below:


As can be seen from the figure, the result of Man (3 bytes) encoding is TWFu (4 bytes), and it is obvious that the volume will increase by 1/3 after Base64 encoding. The string Man is exactly 3 in length and can be represented by four base64 units. But what if the string to be encoded is not an integer multiple of 3?

“If the number of bytes to be encoded is not divisible by 3 and you end up with an extra byte or two, you can do this using the following method: first use a 0 byte value at the end to make it divisible by 3, and then base64 encoding.“

Take the encoded character A as an example, the number of bytes it occupies is 1, which is not divisible by 3, and needs to be supplemented by 2 bytes, as shown in the following figure:


As can be seen from the figure above, the result of character A after base64 encoding is QQ==, and the two = after the result represent the number of complementary bytes. The last 1 base64 byte block has four zeros.

Next, let’s look at another example. Assume that the string to be encoded is BC, which occupies 2 bytes and cannot be divisible by 3, and needs to be supplemented by 1 byte, as shown in the following figure:


As can be seen from the figure above, the base64 encoding result of string BC is QkM=, and the 1 = after the result represents the number of complementary bytes. The last 1 base64 byte block has two zeros.

In JavaScript, there are two functions that handle decoding and encoding base64 strings, respectively:

  • Btoa () : This function creates a Base64-encoded ASCII string based on binary data “strings”.
  • Atob () : This function decodes base64 encoded string data.
1.2.1 BTOA Usage Example
const name = 'Semlinker';
const encodedName = btoa(name);
console.log(encodedName); // U2VtbGlua2Vy
Copy the code
1.2.2 ATOB Usage Examples
const encodedName = 'U2VtbGlua2Vy';
const name = atob(encodedName);
console.log(name); // Semlinker
Copy the code

For both methods, ATOB and BTOA, where A stands for ASCII and B stands for Blob, binary. Thus ATOB stands for ASCII to binary, corresponding to decoding operations. Btoa stands for binary to ASCII, which corresponds to encoding operations. After understanding the meanings of a and B in the method, we will not use them wrong in the future work.

I believe that see here, partners have a certain understanding of Base64. It should be noted that Base64 is only a data encoding method, which aims to ensure the safe transmission of data. But standard Base64 encoding requires no additional information, that is, can be decoded, and is completely reversible. So when it comes to transmitting private data, you can’t use Base64 encoding directly, but use specialized symmetric or asymmetric encryption algorithms.

Two, network download picture -> picture preview

In addition to being able to fetch images locally, we can also use the FETCH API to fetch images from the web and then preview them. Of course, for an image address that is accessible on the network, we can assign the address directly to the IMG element without having to go around the FETCH API. If you need to perform special processing on the image when displaying it, such as decrypting the image data, you can consider using the FETCH API in the Web Worker to fetch the image data and decrypt it.

For simplicity, let’s not consider particular scenarios. First let’s look at the fetch API compatibility:


(Photo credit: https://caniuse.com/#search=fetch)

Then we use the FETCH API to fetch The avatar of Bob from Github, as shown below:


      
<html>
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="Width = device - width, initial - scale = 1.0" />
 <title>Get a remote picture preview example</title>  </head>  <body>  <h3>Po: Get a remote image preview example</h3>  <img id="previewContainer" style="width: 50%;"/>   <script>  const image = document.querySelector("#previewContainer");  fetch("https://avatars3.githubusercontent.com/u/4220799")  .then((response) = > response.blob())  .then((blob) = > {  const objectURL = URL.createObjectURL(blob);  image.src = objectURL;  });  </script>  </body> </html> Copy the code

In the above example, we use the FETCH API to download the avatar of Arbogo from Github. When the request succeeds, we convert the Response object into a Blob object, and then use the url.createObjecturl method. Create an Object URL and assign it to the SRC attribute of the IMG element to display the image.

Using Chrome developer Tools, we can see the Object URL in the Elements panel:


In the green box on the right, we can clearly see that the img element SRC attribute value is a very “special” string:

blob:null/ab24c171-1c5f-4de1-a44e-568bc1f77d7b
Copy the code

The above special string, called “Object URL”, is much simpler than the Data URL introduced earlier. Let’s take a look at the Object URL protocol.

2.1 the Object URL

Object urls are pseudo-protocols, also known as Blob urls. It allows Blob or File objects to be used as URL sources for images, links to download binary data, and so on. In the browser, we create Blob urls using the url.createObjecturl method, which takes a Blob object and creates a unique URL for it of the form Blob :

/

, as shown in the following example:

blob:https://example.org/40a5fb5a-d56d-4a33-b4e2-0acf6a8e5f641
Copy the code

The browser stores a URL β†’ Blob mapping internally for each URL generated with URl.createObjecturl. Therefore, such urls are short, but can access bloBs. The generated URL is valid only if the current document is open. But if the Blob URL you visited no longer exists, you will get a 404 error from your browser.

The Blob URL above may seem like a good idea, but it actually has side effects. Although the URL β†’ Blob mapping is stored, the Blob itself remains in memory and cannot be freed by the browser. The mapping is cleared automatically when the document is unloaded, so the Blob object is then released. However, if the application is long-lived, that won’t happen quickly. Therefore, if we create a Blob URL, it will remain in memory even though the Blob is no longer needed.

For this problem, we can call the url.revokeObjecturl (URL) method to remove references from the internal map, allowing the Blob to be deleted (if there are no other references), and freeing memory.

Since we’re talking Blob urls, we have to mention blobs. So what is a Blob? Let’s move on.

2.2 the Blob API

Blob (Binary Large Object) represents a Large Object of Binary type. In a database management system, the storage of binary data as a single collection of individuals. Blobs are usually video, sound, or multimedia files. “Objects of type Blob in JavaScript represent immutable file-like raw data.” To get a better feel for Blob objects, let’s use the Blob constructor to create a myBlob object, as shown below:


As you can see, the myBlob object has two properties: size and Type. The size attribute is used to represent the size of the data in bytes, and type is a string of MIME type. Blobs don’t necessarily represent data in JavaScript’s native format. For example, the File interface is based on Blob, inheriting Blob functionality and extending it to support files on the user’s system.

Blob consists of an optional string type (usually a MIME type) and blobParts:


❝

MIME (Multipurpose Internet Mail Extensions) is an application type for opening a file with a specified extension name. When the file is accessed, the browser automatically opens the file with a specified application name. It is used to specify the name of a file customized by the client and the opening mode of some media files.

Common MIME types are: hypertext Markup language text. HTML text/ HTML, PNG image. PNG image/ PNG, and plain text.

❞

2.2.1 Blob constructor

The syntax for the Blob constructor is:

var aBlob = new Blob(blobParts, options);
Copy the code

Related parameters are described as follows:

  • BlobParts: It is an array of objects such as ArrayBuffer, ArrayBufferView, Blob, DOMString, etc. DOMStrings will be encoded to UTF-8.
  • Options: An optional object containing the following two properties:
    • Type — The default value is"", which represents the MIME type of the array content to be put into the BLOB.
    • Hamid — The default is"transparent"To specify the inclusion line terminator\nHow the string is written. It is one of two values:"native", indicating that the end-of-line character is changed to a newline suitable for the host operating system file system, or"transparent", which means that the terminator saved in the BLOB is kept unchanged.

“Example 1: Create bloBS from strings“

let myBlobParts = ['<html><h2>Hello Semlinker</h2></html>']; // an array consisting of a single DOMString
let myBlob = new Blob(myBlobParts, {type : 'text/html'.endings: "transparent"}); // the blob

console.log(myBlob.size + " bytes size");
// Output: 37 bytes size
console.log(myBlob.type + " is the type"); // Output: text/html is the type Copy the code

“Example 2: Create bloBs from typed arrays and strings“

let hello = new Uint8Array([72.101.108.108.111]); // δΊŒθΏ›εˆΆζ ΌεΌηš„ "hello"
let blob = new Blob([hello, ' '.'semlinker'] and {type: 'text/plain'});
Copy the code

With the Blob constructor out of the way, let’s look at the properties and methods of the Blob class:


2.2.2 Blob properties

We already know that a Blob object contains two properties:

  • Size (read-only) : indicates the valueBlobThe size, in bytes, of the data contained in the object.
  • Type (read only) : A string indicating theBlobObject contains the MIME type of the data. If the type is unknown, the value is an empty string.
2.2.3 Blob method
  • Slice ([start[, end[, contentType]]]) : Returns a new Blob containing the data in the range specified in the source Blob.
  • Stream () : Returns a ReadableStream that can read bloB content.
  • Text () : UtF-8 USVString that returns a Promise object containing all the contents of the BLOB.
  • ArrayBuffer () : An arrayBuffer that returns a Promise object in binary format containing all the contents of the BLOB.

One thing to note here is that “Blob objects are immutable”. We can’t change data directly in a Blob, but we can split a Blob, create new Blob objects from it, and blend them into a new Blob. This behavior is similar to JavaScript strings: we cannot change characters in strings, but we can create new, corrected strings.

For the Response object of the FETCH API, in addition to the BLOb () method, the object also provides json(), text(), formData(), and arrayBuffer() methods to convert the Response into different data formats.

In projects where the front and back ends are separated, the json() method is likely to be used more often than other methods. For the previous example, we converted the response object into an ArrayBuffer, which can also display images downloaded from the network normally, as shown below:

<h3>Po: Get a remote image preview example</h3>
<img id="previewContainer" style="width: 50%;"/>

<script>
 const image = document.querySelector("#previewContainer");  fetch("https://avatars3.githubusercontent.com/u/4220799")  .then((response) = > response.arrayBuffer())  .then((buffer) = > {  const blob = new Blob([buffer]);  const objectURL = URL.createObjectURL(blob);  image.src = objectURL;  }); </script> Copy the code

In the above code, we first convert the response Object to an ArrayBuffer, then convert the ArrayBuffer Object to a Blob by calling the Blob constructor, and create the ObjectURL using the createObjectURL() method. Finally realize the image preview.

For those of you who are not familiar with ArrayBuffer, let’s take a look at its mystery.

2.3 with TypedArray ArrayBuffer

2.3.1 ArrayBuffer

The ArrayBuffer object is used to represent a “generic, fixed-length” buffer of raw binary data. “An ArrayBuffer does not operate directly, but through a type array object or DataView object,” which represents the data in the buffer in a specific format and reads and writes the contents of the buffer in those formats.

❝

An ArrayBuffer is simply a piece of memory, but you can’t use it directly. This is just as if you were in C and malloc was a piece of memory, you would also convert it to unsigned_int32 or int16, whatever array/pointer of the actual type you need to use.

That’s what TypedArray does in JS. All of those Uint32Array and Int16Array provide a “View” to the ArrayBuffer, MDN is called Multiple views on the same data. Subscript reading and writing to them will eventually be reflected on the ArrayBuffer it is built on.

Source: https://www.zhihu.com/question/30401979

❞

“grammar“

new ArrayBuffer(length)
Copy the code
  • Parameter: length indicates the size, in bytes, of the ArrayBuffer to be created.
  • Return value: an ArrayBuffer of a specified size whose contents are initialized to 0.
  • Exception: If length is greater than number.max_SAFE_INTEGER (>= 2 ** 53) or negative, a RangeError exception is thrown.

“The sample“

The following example creates an 8-byte buffer and references it with an Int32Array:

let buffer = new ArrayBuffer(8);
let view   = new Int32Array(buffer);
Copy the code

As of ECMAScript 2015, ArrayBuffer objects need to be created with the new operator. If the constructor is called without new, TypeError is thrown. For example, executing let ab = ArrayBuffer(10) raises the following exception:

VM109:1 Uncaught TypeError: Constructor ArrayBuffer requires 'new'
    at ArrayBuffer (<anonymous>)
    at <anonymous>:1:10
Copy the code

Some common Web apis, such as FileReader API and Fetch API, also support ArrayBuffer at the bottom. Here we take FileReader API as an example. See how to read a File object as an ArrayBuffer:

const reader = new FileReader();

reader.onload = function(e) {
  let arrayBuffer = reader.result;
}
 reader.readAsArrayBuffer(file); Copy the code
2.3.2 Unit8Array

Uint8Array array type represents an array of 8-bit unsigned integers that are initialized to 0 when created. Once created, you can refer to the elements of an array “as objects or by using an array subscript index.”

“grammar“

new Uint8Array(a);// ES2017 latest syntax
new Uint8Array(length); // Create an array of unsigned integers with length initialized to 0
new Uint8Array(typedArray);
new Uint8Array(object);
new Uint8Array(buffer [, byteOffset [, length]]);
Copy the code

“The sample“

// new Uint8Array(length); 
var uint8 = new Uint8Array(2);
uint8[0] = 42;
console.log(uint8[0]); / / 42
console.log(uint8.length); / / 2
console.log(uint8.BYTES_PER_ELEMENT); / / 1  // new TypedArray(object);  var arr = new Uint8Array([21.31]); console.log(arr[1]); / / 31  // new Uint8Array(typedArray); var x = new Uint8Array([21.31]); var y = new Uint8Array(x); console.log(y[0]); / / 21  // new Uint8Array(buffer [, byteOffset [, length]]); var buffer = new ArrayBuffer(8); var z = new Uint8Array(buffer, 1.4); Copy the code
2.3.3 Relationship between ArrayBuffer and TypedArray

The ArrayBuffer itself is just a row of zeros and ones. The ArrayBuffer does not know the separation position between the first and second elements in the array.


(Image credit — A Cartoon Intro to ArrayBuffers and SharedArrayBuffers)

To provide context, and actually break it down into boxes, we need to wrap it in what’s called a view. You can add these data views using type arrays, and you can use many different types of type arrays.

For example, you can have an array of type Int8, which will divide the array into 8-bit byte arrays.


(Image credit — A Cartoon Intro to ArrayBuffers and SharedArrayBuffers)

Or you can have an unsigned Int16 array, which splits the array into 16-bit byte arrays and treats it as an unsigned integer.


(Image credit — A Cartoon Intro to ArrayBuffers and SharedArrayBuffers)

You can even have multiple views on the same base buffer. Different views give different results for the same operation.

For example, if we get the value of the 0 & 1 element (-19 & 100) from the Int8 view of this ArrayBuffer, it will give us a different value from the 0 (25837) element in the Uint16 view, even though they contain exactly the same bits.


(Image credit — A Cartoon Intro to ArrayBuffers and SharedArrayBuffers)

In this way, an ArrayBuffer is basically like raw memory. It simulates direct memory access using languages such as C. You may be wondering why we added this abstraction layer instead of giving programs direct access to memory, which would cause some security holes.

Now that we know about bloBs and ArrayBuffers, how were they different before?

2.4 the Blob vs ArrayBuffer

The ArrayBuffer object is used to represent a generic, fixed-length buffer of raw binary data. Instead of manipulating the contents of an ArrayBuffer directly, you need to create a typed array object or DataView object that represents the buffer in a specific format, and use that object to read and write the contents of the buffer.

Objects of type “Blob” represent raw data of immutable file-like objects. Blobs don’t necessarily represent data in JavaScript’s native format. The File interface is based on Blob, inheriting Blob functionality and extending it to support files on the user’s system.

2.4.1 Differences between Blob and ArrayBuffer
  • Unless you need to use the write/edit capabilities provided by ArrayBuffer, the Blob format is probably best.
  • Blob objects are immutable, whereas arrayBuffers can be manipulated via TypedArrays or DataViews.
  • An ArrayBuffer is stored in memory and can be manipulated directly. Blobs can be on disk, cache memory, and other locations that are not available.
  • Although blobs can be passed directly as arguments to other functions, for examplewindow.URL.createObjectURL(). However, you may still need a File API like FileReader to work with blobs.
  • Blob and ArrayBuffer objects are interchangeable:
    • The use of FileReaderreadAsArrayBuffer()Method to convert a Blob object into an ArrayBuffer object;
    • Use the Blob constructor, as innew Blob([new Uint8Array(data]);To convert an ArrayBuffer object into a Blob object.

To help you understand the conversion process, Po Ge gives a simple example of mutual conversion:

2.4.2 Blob converted to ArrayBuffer
var blob = new Blob(["\x01\x02\x03\x04"]),
    fileReader = new FileReader(),
    array;

fileReader.onload = function() {
 array = this.result;  console.log("Array contains", array.byteLength, "bytes."); };  fileReader.readAsArrayBuffer(blob); Copy the code
2.4.3 ArrayBuffer Blob
var array = new Uint8Array([0x01.0x02.0x03.0x04]);
var blob = new Blob([array]);
Copy the code

2.5 DataView ArrayBuffer

DataView is a low-level interface that can read and write multiple numeric types from binary ArrayBuffer objects, regardless of platform endianness.

❝

Byte order, also called Endianness or endorder (English: Endianness), in computer science, refers to the order of the bytes that make up multibyte words in memory or on a digital communication link.

There are two general rules for how bytes are arranged. For example, if the least significant byte (similar to the least significant bit) of a multi-bit integer precedes the most significant byte among the bytes sorted from lowest to highest by storage address, it is called little endian; Otherwise, it is called the large endian sequence. In network applications, byte order is a factor that must be considered, because different machine types may use different standard byte order, so all conversions follow network standards.

For example, if the above variable x is of type int and is located at address 0x100, its value is 0x01234567 and the address range is 0x100 to 0x103 bytes, and its internal order depends on the type of machine. The big-endian method will start with the first digit: 0x100:01, 0x101:23,.. . The small endian method will be: 0x100:67, 0x101:45,.. .

❞

2.5.1 DataView Constructor
new DataView(buffer [, byteOffset [, byteLength]])
Copy the code

Related parameters are described as follows:

  • Buffer: An existing ArrayBuffer or SharedArrayBuffer object, the data source for the DataView object.
  • ByteOffset (optional) : byteOffset in buffer of the first byte of this DataView object. If not specified, the first byte is started by default.
  • ByteLength: The length of this DataView object in bytes. If not specified, the length of the view will match the length of the buffer.

“DataView return value“

Calling the DataView constructor with new returns a new DataView object representing the specified data cache. You can think of the returned object as an “interpreter” for a binary byte buffer array buffer — it knows how to translate the bytecode correctly when it is read or written. This means that it handles integer and floating point conversions, byte order, and other details at the binary level.

“DataView Example“

const buffer = new ArrayBuffer(16);

// Create a couple of views
const view1 = new DataView(buffer);
const view2 = new DataView(buffer, 12.4); //from byte 12 for the next 4 bytes
view1.setInt8(12.42); // put 42 in slot 12  console.log(view2.getInt8(0)); // expected output: 42 Copy the code
2.5.2 DataView properties

All DataView instances inherit from dataView. prototype and allow you to add additional properties to a DataView object.

  • DataView. Prototype. Buffer (read only) : to create the DataView set ArrayBuffer object;
  • DataView. Prototype. ByteLength (read only) : it means ArrayBuffer of bytes or SharedArrayBuffer objects;
  • DataView. Prototype. ByteOffset (read only) : it means when reading from an ArrayBuffer offset bytes.
2.5.3 DataView method

DataView objects provide methods such as getInt8(), getUint8(), setInt8(), and setUint8() to manipulate data. We will not go into details about the use of each method. Here’s a simple example:

const buffer = new ArrayBuffer(16);
const view = new DataView(buffer, 0);

view.setInt8(1.68);
view.getInt8(1); / / 68
Copy the code

After introducing ArrayBuffer, TypedArray, and DataView, Arbog uses a diagram to summarize the relationship between them.


Ok, let’s move on to the next step.

Three, the picture grayscale

To grayscale the image, we need to manipulate the image pixel data. So how do we get pixel data from an image?

3.1 getImageData method

To solve this problem, we can use getImageData provided by CanvasRenderingContext2D to retrieve image pixel data. GetImageData () returns an ImageData object, It is used to describe the pixel data hidden in the Canvas area, which is represented by a rectangle with the starting point (sx, SY), width (SW) and height (sh). The syntax of getImageData is as follows:

ctx.getImageData(sx, sy, sw, sh);
Copy the code

The parameters are described as follows:

  • Sx: The upper-left x-coordinate of the rectangular region of the image data to be extracted.
  • Sy: Y coordinates of the upper-left corner of the rectangular region of the image data to be extracted.
  • Sw: Width of the rectangular area of image data to be extracted.
  • Sh: height of the rectangle region of image data to be extracted.

3.2 putImageData method

After obtaining the pixel data of the picture, we can process the obtained pixel data, such as grayscale or invert color processing. When the processing is complete, to display the processing effect on the page, we need to make use of putImageData, another API provided by CanvasRenderingContext2D.

This API is the Canvas 2D API’s way of drawing data from an existing ImageData object. If a drawn rectangle is provided, only the pixels of that rectangle are drawn. This method is not affected by the canvas transformation matrix. The syntax of the putImageData method is as follows:

void ctx.putImageData(imagedata, dx, dy);
void ctx.putImageData(imagedata, dx, dy, dirtyX, dirtyY, dirtyWidth, dirtyHeight);
Copy the code

The parameters are described as follows:

  • ImageData: imageData, an array object containing pixel values.
  • Dx: The offset (X-axis offset) of the source image data in the destination canvas.
  • Dy: The offset of the position of the source image data in the target canvas (the offset in the y direction).
  • DirtyX (Optional) : Position of the upper-left corner of the rectangle in the source image data. The default is the upper left corner (x-coordinate) of the entire image data.
  • DirtyY (Optional) : Position of the upper-left corner of the rectangle in the source image data. The default is the upper left corner (y-coordinate) of the entire image data.
  • DirtyWidth (Optional) : Width of the rectangular region in the source image data. The default is the width of the image data.
  • DirtyHeight (Optional) : Height of the rectangular region in the source image data. The default is the height of image data.

3.3 Image grayscale processing

After introducing the getImageData() and putImageData() methods, let’s see how they can be used to grayscale images:


      
<html>
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="Width = device - width, initial - scale = 1.0" />
 <title>Get remote image and grayscale</title>  </head>  <body>  <h3>Po: Get a remote image and grayscale the example</h3>  <div>  <button id="grayscalebtn">Gray,</button>  <div style="display: flex;">  <div style="flex: 50%;">  <p>Preview the container</p>  <img  id="previewContainer"  width="230"  height="230"  style="border: 2px dashed blue;"  />  </div>  <div style="flex: 50%;">  <p>The Canvas container</p>  <canvas  id="canvas"  width="230"  height="230"  style="border: 2px dashed grey;"  ></canvas>  </div>  </div>  </div>  <script>  const image = document.querySelector("#previewContainer");  const canvas = document.querySelector("#canvas");   fetch("https://avatars3.githubusercontent.com/u/4220799")  .then((response) = > response.blob())  .then((blob) = > {  const objectURL = URL.createObjectURL(blob);  image.src = objectURL;  image.onload = (a)= > {  draw();  };  });   function draw() {  const ctx = canvas.getContext("2d");  ctx.drawImage(image, 0.0.230.230);  const imageData = ctx.getImageData(0.0, canvas.width, canvas.height);  const data = imageData.data;   const grayscale = function () {  for (let i = 0; i < data.length; i += 4) {  const avg = (data[i] + data[i + 1] + data[i + 2) /3;  data[i] = avg; // red  data[i + 1] = avg; // green  data[i + 2] = avg; // blue  }  ctx.putImageData(imageData, 0.0);  };  const grayscalebtn = document.querySelector("#grayscalebtn");  grayscalebtn.addEventListener("click", grayscale);  }  </script>  </body> </html> Copy the code

In the example above, we first download the avatar of Bob from Github, then preview it locally, then call draw() to draw the image to the Canvas container of the page, and bind the listener event to the grayscale button.

When the user clicks the grayscale button, the grayscale processing function will be triggered. Inside the function, the image pixels obtained through the ctx.getimageData () method will be grayscale processed. After the processing is complete, the processed pixel data is updated to Canvas through ctx.putimageData ().

After the above code is successfully run, the final grayscale effect is shown as the figure below:

Four, picture compression

In some cases, we want to compress the image before submitting it to the server to reduce the amount of data transferred. To achieve image compression on the front end, we can use the toDataURL() method provided by the Canvas object, which takes two optional parameters, Type and encoderOptions.

Type indicates the image format. The default value is image/ PNG. EncoderOptions is used to indicate the quality of the image. If the image format is image/ JPEG or image/webp, the quality of the image can be selected from 0 to 1. If the value is outside the range, the default value of 0.92 will be used and other parameters will be ignored.

Let’s look at how to compress the image that has been grayed previously.

<button id="compressbtn">Image compression</button>
<div style="display: flex;">
   <div style="flex: 33.3%;">
      <p>Preview the container</p>
      <img id="previewContainer" width="230" height="230"  style="border: 2px dashed blue;" />  </div>  <div style="flex: 33.3%;">  <p>The Canvas container</p>  <canvas id="canvas" width="230" height="230"  style="border: 2px dashed grey;">  </canvas>  </div>  <div style="flex: 33.3%;">  <p>Compressed preview container</p>  <img id="compressPrevContainer" width="230" height="230"  style="border: 2px dashed green;" />  </div> </div>  <script>  const compressbtn = document.querySelector("#compressbtn");  const compressImage = document.querySelector("#compressPrevContainer");  compressbtn.addEventListener("click", compress);   function compress(quality = 80, mimeType = "image/webp") {  const imageDataURL = canvas.toDataURL(mimeType, quality / 100);  compressImage.src = imageDataURL;  } </script> Copy the code

In the above code, we set the default image quality to be “0.8” and the image type to be “image/webp”. When the user clicks the compress button, the toDataURL() method of Canvas object will be called to compress the image. After the above code is successfully run, the final processing effect is shown as the figure below:


In addition to the toDataURL() method, Canvas also provides a toBlob() method, which has the following syntax:

canvas.toBlob(callback, mimeType, qualityArgument)
Copy the code

Compared to toDataURL(), the toBlob() method is asynchronous and therefore has a callback argument. The default first argument to this callback method is converted BLOB file information.

5. Upload pictures

After obtaining the Data URL Data corresponding to the compressed image, the Data can be submitted directly to the server. In this case, the server needs to do some related processing to save the uploaded image normally. Here, Express is used as an example. The specific processing code is as follows:

const app = require('express') ();
app.post('/upload'.function(req, res){
    let imgData = req.body.imgData; // Get base64 image data in the POST request
    let base64Data = imgData.replace(/^data:image\/\w+; base64,/."");
 let dataBuffer = Buffer.from(base64Data, 'base64');  fs.writeFile("abao.png", dataBuffer, function(err) {  if(err){  res.send(err);  }else{  res.send("Picture uploaded successfully!");  }  }); }); Copy the code

However, the returned image Data in Data URL format is generally large. To further reduce the amount of Data transferred, we can convert it into a Blob object:

function dataUrlToBlob(base64, mimeType) {
  let bytes = window.atob(base64.split(",") [1]);
  let ab = new ArrayBuffer(bytes.length);
  let ia = new Uint8Array(ab);
  for (let i = 0; i < bytes.length; i++) {
 ia[i] = bytes.charCodeAt(i);  }  return new Blob([ab], { type: mimeType }); } Copy the code

After the transformation is complete, the Blob object corresponding to the compressed image can be wrapped in a FormData object and then submitted to the server via AJAX:

function uploadFile(url, blob) {
  let formData = new FormData();
  let request = new XMLHttpRequest();
  formData.append("imgData", blob);
  request.open("POST", url, true);
 request.send(formData); } Copy the code

Six, Po Ge has words

6.1 How Can I View binary Image Data

To see the binary data for the image, you need to use some off-the-shelf editor, such as the “WinHex” on Windows or the “Synalyze It! Pro” hexadecimal editor on macOS. Here we use Synalyze It! Pro this editor, in hexadecimal form to view the binary data corresponding to the head of Po Elder brother.


6.2 How to distinguish the Types of pictures

“Computers do not discriminate between image types by their name suffixes, but by ‘Magic numbers’.” For some types of files, the first few bytes are fixed, and the file type can be determined based on the contents of these bytes.

The magic numbers corresponding to common image types are shown in the following table:

The file type The file suffix The magic number
JPEG jpg/jpeg 0xFFD8FF
PNG png 0x89504E47
GIF gif 0 x47494638 (GIF8)
BMP bmp 0x424D

Here we use the image of Abao.png as an example to verify that the image type is correct:


In the daily development process, if we encounter a scenario to detect image types, we can directly use some ready-made third-party libraries. For example, if you want to determine if an image is PNG, you can use the IS-png library, which supports both browser and Node.js, as shown in the following example:

“Node.js“

// npm install read-chunk
const readChunk = require('read-chunk'); 
const isPng = require('is-png');
const buffer = readChunk.sync('unicorn.png'.0.8);

isPng(buffer); //=> true Copy the code

“Browser“

(async() = > { const response = await fetch('unicorn.png');
 const buffer = await response.arrayBuffer();

 isPng(new Uint8Array(buffer));
 //=> true }) ();Copy the code

6.3 How do I Obtain the Image size

The size, bit depth, color type and compression algorithm of the image are all stored in the binary data of the file. Let’s continue to use the image of Abao.png as an example to understand the actual situation:


❝

528 (decimal) => 0x0210 (hexadecimal)

560 (decimal) => 0x0230 (hexadecimal)

❞

So if we want to get the size of the image, we need to parse the image binary data according to different image formats. Fortunately, we don’t need to do this ourselves. The image-size node. js library already provides the ability to get file sizes for major image types, as shown in the following example:

“synchronously“

var sizeOf = require('image-size');

var dimensions = sizeOf('images/abao.png');
console.log(dimensions.width, dimensions.height);
Copy the code

“asynchronous“

var sizeOf = require('image-size');

sizeOf('images/abao.png'.function (err, dimensions) {
  console.log(dimensions.width, dimensions.height);
});
Copy the code

Image-size is a powerful library that supports BMP, GIF, ICO, JPEG, SVG and WebP formats in addition to PNG.

6.4 How to decode pixel Data in PNG Images

I believe that boys usually also heard picture decoding, audio and video decoding. Decoding a PNG image converts an image from binary data into ImageData, which contains pixel data. As mentioned earlier, the getImageData() method provided by CanvasRenderingContext2D can be used to retrieve image pixel data.

So what happens inside the getImageData() method? Let’s briefly introduce the general process. Here we take a 2px * 2px picture as an example, and the following is the enlarged display effect:

(photo: https://vivaxyblog.github.io/2019/12/07/decode-a-png-image-with-javascript-cn.html)

Again, we first use the “Synalyze It! Pro” hexadecimal editor to open the “2px by 2px” image above:


The pixel data of a PNG image is stored in the “IDAT” block, which contains other data blocks besides the “IDAT” block. The complete data block is shown as follows:


(photo: https://dev.gameres.com/Program/Visual/Other/PNGFormat.htm)

Before parsing pixel data, let’s look at how pixel data is encoded. Each row of pixels is first processed by a filter function, which can be different for each row of pixels. All rows of pixel data are then compressed by the “Deflate” compression algorithm. Here Po uses the pako library to decode operations:

const pako = require("pako");

const compressed = new Uint8Array([120.156.99.16.96.216.0.0.0.228.0.193]);
try {
  const result = pako.inflate(compressed);
 console.dir(result); } catch (err) {  console.log(err); } Copy the code

In the code above, the decompression is performed by calling the pako.inflate() method, resulting in the uncompressed pixel data as follows:

Uint8Array [ 0, 16, 0, 176 ]
Copy the code

After extracting the extracted pixel data, the scan line is decoded and the pixel information of the image is restored according to the index in the color palette. Here Po brother will not expand in detail, interested partners can read step by step decoding PNG images this article.

6.5 How Can I Upload Large File Fragments

File objects are special types of BLOBs and can be used in the context of any Blob type. Therefore, in the scenario of large file transmission, we can use the slice method to cut large files and then slice them for uploading. The specific example is as follows:

const file = new File(["a".repeat(1000000)]."test.txt");

const chunkSize = 40000;
const url = "https://httpbin.org/post";

async function chunkedUpload() {  for (let start = 0; start < file.size; start += chunkSize) {  const chunk = file.slice(start, start + chunkSize + 1);  const fd = new FormData();  fd.append("data", chunk);   await fetch(url, { method: "post".body: fd }).then((res) = >  res.text()  );  } } Copy the code

6.6 How Can I Download files

In some scenarios, we use Canvas for image editing or jsPDF, SheetJS and other third-party libraries for document processing. When the file processing is complete, we need to download and save the file locally. For these scenarios, we can use a pure front-end solution to implement file downloads.

“Talk is cheap”, a simple Blob file download example:

“index.html“


      
<html>
  <head>
    <meta charset="UTF-8" />
    <title>Blob file download example</title>
 </head>   <body>  <button id="downloadBtn">File download</button>  <script src="index.js"></script>  </body> </html> Copy the code

“index.js“

const download = (fileName, blob) = > {
  const link = document.createElement("a");
  link.href = URL.createObjectURL(blob);
  link.download = fileName;
  link.click();
 link.remove();  URL.revokeObjectURL(link.href); };  const downloadBtn = document.querySelector("#downloadBtn"); downloadBtn.addEventListener("click", (event) => {  const fileName = "blob.txt";  const myBlob = new Blob(["A thorough grasp of Blob Web API"] and {type: "text/plain" });  download(fileName, myBlob); }); Copy the code

In our example, we create a Blob object of type “text/plain” by calling the Blob’s constructor, and then download the file by dynamically creating the A tag. In the actual project development process, we can use mature open source library, such as Filesaver.js to realize the file saving function.

7. Reference resources

  • PNGFormat
  • w3.org – PNG
  • Wiki – byte order
  • Decoding PNG images step by step
  • comprehensive-image-processing-on-browsers

❞


This article is formatted using MDNICE