This is the 28th day of my participation in the August Text Challenge.More challenges in August

1. Service scenario of file uploading

In addition to the text character type data we normally submit, many times we submit data in other formats such as images, audio, video, and so on.

  • Upload user profile picture
  • Upload web disk files
  • …

2. Client data submission

Regardless of the client environment (e.g., browser, Postman, Thunderbolt, or whatever), the most basic and important thing is the protocol used by the client and server.

2-1. Send the request using a browser

As mentioned above, to send a request is to use the above canonical format, link to the server and submit some data (resources) and get the data (resources) returned by the server. These contents include:

agreement

How and in what format data resources interact.

URL

Location of the data resource

Request method

What to do with this data resource: get? To add? Modified? Replace? To delete?

Header information

Additional description (metadata) of the resource request.

The body of the

The actual data that needs to be submitted when adding, modifying, or replacing data.

2-2. Submit the request using a browser

When we use the address bar provided by the browser to send a request, we can only set the URL in it, other parts of the browser default:

  • Request method:GET 。
  • Header information: Automatically set by the browser (varies by browser).
  • Text: According toHTTPProtocol Specification DefinitionGETThe body cannot be submitted.

Only simple GET requests can be sent in the address bar of a browser.

2-3. Submit the request using a form in HTML

Forms in HTML provide more request Settings than the browser itself.

<form action="/postData" method="post" enctype="application/x-www-form-urlencoded">
  <p>Name:<input type="text" name="username" />
  </p>
  <p>
    <button>submit</button>
  </p>
</form>
Copy the code

action

Request the address

method

Request method, form only support: GET, POST.

enctype

The format of the data in the request body corresponds to: content-type in the request header. There are three values:

  • Application/X-www-form-urlencoded: URL format coded data,enctypeThe default value of.
  • Multipart /form-data: Array (raw data) encoded in form-data format – used when uploading files.
  • Text /plain: indicates plain text data.

Note:

Content-type Specifies the Type of data, but does not necessarily indicate that the data is in that format. Such as: Your data is organized in JSON format, but you can also set the content-type to text/ HTML. Generally, the receiver uses the content-type to identify the Type and call the corresponding method to parse it. So if the Content-Type doesn’t match the actual Content format, it can cause the receiver to parse incorrectly.

3. Webserver receives and processes the submitted data

When the server receives the request, it actually obtains all the data related to the request. Net and HTTP modules in Node.js encapsulate the parsing of these data and provide the corresponding API to obtain them:

//C3-0. server.on('request'.(req, res) = > {
  console.log('Request:', req.method);
  console.log('HTTP Protocol Version ', req.httpVersion);
  console.log('Request URL:', req.url);
  
  console.log('Request Header:', req.headers); })...Copy the code

Based on the structure defined by the HTTP standard specification, we usually conduct some data interaction between client and server in the following manner.

3-1, URL

The URL requested by the client is itself a kind of data.

//C3-1-0. server.on('request'.async (req, res) => {
  // The requested URL is a set of data. The server can do different things with different URL data and return different results
  if (req.url == '/') {
    res.end('index');
  }

  if (req.url == '/register') {
    res.end('register');
  }
  
  // Dynamic routing
  let dyRouter = /\/item\/(? 
      
       \d+)/gi
      .exec(req.url);
  if (dyRouter) {
    let { id } = dyRouter.groups;

    res.end(`id: ${id}`); }}); .Copy the code
// Use koa & koa-router
// C3-1-1-1. router.get('/item/:id(\d+)'.async (ctx, next) => {
  let {id} = ctx.params;
  
  res.end(`id: ${id}`); })...Copy the code

In the 3-1-2 s, the QueryString

QueryString is in the URL, right? For the rest, additional arguments are provided to the server using an ampersand separated list of key/value pairs, such as:? Key1 = value1 & key2 = value2. The server can use these parameters to perform additional operations.

// C3-1-2-0.// http://localhost:8888/items? page=1&limit=5
server.on('request'.(req, res) = > {
    const u = new URL(`${req.headers.host}${req.url}`);
    const query = u.searchParams;
    console.log(`page: ${query.get('page')} , limit: ${query.get('limit')}`); }); .Copy the code
// C3-1-2-1.// http://localhost:8888/items? page=1&limit=5
router.get('/item'.async (ctx, next) => {
  let {page, limit} = ctx.query;
  
  console.log(`page: ${page} , limit: ${limit}`); })...Copy the code

QueryString is part of the URL, so it has nothing to do with the request method, i.e., whatever the request is (GET/POST/PUT/….). You can have a queryString.

3-2, the text

The main data for the body request and response. In the HTTP protocol standard, body data is not always portable:

Reference: developer.mozilla.org/zh-CN/docs/…

Body data often involves complex data, so the sender needs to organize the data in a format that tells the receiver the MIME Type (organization structure) of the data through the content-Type header.

application/x-www-form-urlencoded

A data format encoded in URLEncode, used in QueryString.

Reference: www.eso.org/~ndelmott/u…

application/json

As the name implies, it is in JSON format.

text/plain

If the data does not require any processing (even if it is JSON-structured), the recipient can be told via Text /plain.

multipart/form-data

Some of the above types are based on character data, while others contain not only characters but other types, such as binary data. At this point, we can format the data with form-data and declare the content-Type as multipart/form-data; A boundary = random.

Reference: developer.mozilla.org/zh-CN/docs/…

3-2-1, node.js based native text parsing processing

// C3-2-1-0
const Koa = require('koa');
const KoaRouter = require('koa-router');
const queryString = require('querystring');

const server = new Koa();
const router = new KoaRouter();

// Body parsing middleware
const koaBody = () = > {
  return (ctx, next) = > {

    return new Promise((resolve, reject) = > {
      let data = ' ';

      // ctx.req => the IncomingMessage object of the HTTP module in Node.js
      // Data event: continuously emitted during receiving data
      // chunk: received binary buffer data streams
      ctx.req.on('data'.async (chunk) => {
        // Concatenate the received data
        data += chunk.toString();
      });

      // end: data receiving is triggered
      ctx.req.on('end'.async() = > {Ctx. is a method provided by the wrapper under the Reuqest object in Koa,
        // Verifies that the value of 'content-type' in the current request is one of the values specified in the parameter
        // Content-Type: application/json
        // ctx.is(['application/json', 'application/x-www-form-urlencoded']) returns Application /json
        if (ctx.is('application/json')) {
          // Perform JSON parsing on the data
          ctx.request.body = JSON.parse(data);
        } else {
          // QueryString+urldecoded parse the data
          ctx.request.body = queryString.parse(data);
        } else {
          // If the above processing is not met, the original string is used by default
          ctx.request.body = data;
        }
        // Use the above parsing to save the parsed results to ctx.request.body for subsequent middleware calls
        // The same is true of the basic principles of the KOA-body middleware

        resolve();
      });
    }).then((res) = > {
      return next();
    })

  }
}

router.post('/user', koaBody(), async (ctx, next) => {
    console.log('body', ctx.request.body);
    ctx.body = 'user';

});

server.use(router.routes());

server.listen(8888);
Copy the code

3-2-2, use of KOA-body middleware in Koa

In KOA, the koA-body middleware is used to handle the parsing of body content

Reference: www.npmjs.com/package/koa…

// C3-2-2-0
const Koa = require('koa');
const KoaRouter = require('koa-router');
const koaBody = require('koa-body');

const server = new Koa();
const router = new KoaRouter();

router.post('/user', koaBody({
  / / koa - body configuration
}), async (ctx, next) => {
    ctx.body = 'user';
});

server.use(router.routes());

server.listen(8888);
Copy the code

Koa-body configuration description

  • patchNode {Boolean} Patch request body to Node’s ctx.req, default false

    • Whether to store the parsed content toctx.reqProperty, the default is:false.

  • patchKoa {Boolean} Patch request body to Koa’s ctx.request, default true

    • Whether to store the parsed content toctx.requestProperty, the default is:true.

  • jsonLimit {String|Integer} The byte (if integer) limit of the JSON body, default 1mb

    • Set up theJSONFormat Data size. Supports both numeric and string values, such as'1kb' 或 100If the value is a number, the unit isbyte, the default is:1mb 。

  • formLimit {String|Integer} The byte (if integer) limit of the form body, default 56kb

    • withjsonLimitTo set upapplication/x-www-form-urlencodedValue size, default56kb 。

  • textLimit {String|Integer} The byte (if integer) limit of the text body, default 56kb

    • withjsonLimitTo set uptext/plainValue size, default56kb 。

  • encoding {String} Sets encoding for incoming form fields, default utf-8

    • Set request data encoding, defaultutf-8 。

  • multipart {Boolean} Parse multipart bodies, default false

    • Whether to enable parsing multipart/form-data. The default value is false.

    • Note: This defaults to false, so if you want to use koa-body to process multipart/form-data data, remember to turn this on.

  • urlencoded {Boolean} Parse urlencoded bodies, default true

    • Enabled or notapplication/x-www-form-urlencodedData parsing, default istrue 。

  • text {Boolean} Parse text bodies, such as XML, default true

    • Enabled or nottext/plainData parsing, default istrue 。

  • json {Boolean} Parse JSON bodies, default true

    • Enabled or notapplicatin/jsonData parsing, default istrue 。

  • jsonStrict {Boolean} Toggles co-body strict mode; if set to true – only parses arrays or objects, default true

    • Whether to enableco-bodyStrict parsing mode if set totrueIs only processedarrays 和 objectsBy default,true

  • includeUnparsed {Boolean} Toggles co-body returnRawBody option; if set to true, for form encoded and JSON requests the raw, unparsed requesty body will be attached to ctx.request.body using a Symbol, default false

    • Whether to append the original content for parsing toctx.request.bodyOn the properties (encoded 和 json, the default isfalse 。

  • formidable {Object} Options to pass to the formidable multipart parser

    • Set file upload options, object format.
    • koa-bodyOther third-party libraries are usednode-formidableImplement file upload, so this setting is set for this third-party library.
    • You need to set bothmultipartOptions fortrue 。

  • onError {Function} Custom error handle, if throw an error, you can customize the response – onError(error, context), default will throw

    • Custom error handlers.

  • parsedMethods {String[]} Declares the HTTP methods where bodies will be parsed, default ['POST', 'PUT', 'PATCH'].

    • Set up thekoa-bodyWhat will it behttpRequest method, the default is['POST', 'PUT', 'PATCH'] 。

Formidable configuration

By default, results for file types are stored in the ctx.request.files property, and results for non-file types (characters) are stored in ctx.request.body.

  • maxFields {Integer} Limits the number of fields that the querystring parser will decode, default 1000

    • Set up thequeryStringMaximum number of fields, default1000 。

  • maxFieldsSize {Integer} Limits the amount of memory all fields together (except files) can allocate in bytes. If this value is exceeded, an ‘error’ event is emitted, default 2mb (2 * 1024 * 1024)

    • Set the maximum content size for non-file data. Default2mb 。

  • uploadDir {String} Sets the directory for placing file uploads in, default os.tmpDir()

    • Upload a directory for saving files. The default for the OS. TmpDir ()

    • OS is the built-in system module of Node.js. The tmpDir() method is used to obtain the system temporary directory under the current system.

  • keepExtensions {Boolean} Files written to uploadDir will include the extensions of the original files, default false

    • Indicates whether to retain the suffix (extension) of the uploaded file. The default value isfalse 。

  • hash {String} If you want checksums calculated for incoming files, set this to either 'sha1' or 'md5', default false

    • Whether to verify the fingerprint of uploaded files. The default value isfalse 。

  • multiples {Boolean} Multiple file uploads or no, default true

    • Whether multiple files can be uploaded. The default value istrue 。

  • onFileBegin {Function} Special callback on file begin. The function is executed directly by formidable. It can be used to rename files before saving them to disk.

    • File upload event callback function, which can be used with the return value to change the name of the uploaded file (default uploaded file is usedhashTo name it.

Reference: github.com/node-formid…

4. What is stateless in HTTP? What’s wrong with it?

4-1. Stateless

The so-called state can be understood as the memory of transaction processing. Stateless means no memory. If there is a need to have a state, the request needs to be maintained in the memory of the request, which will bring burden. Meanwhile, the real-time requirements of the Web are not as high as real-time chat, voice and video calls or games, which will lead to the waste of server resources.

One problem with this, of course, is the inability to process associated transactions, such as users who are identified in some requests but not in the next.

4-2, have a conversation

Session refers to a communication process between the client and the server. Although stateless feature does not proactively maintain the state of the session in real time, we can actively establish some associations in each session. The cookie mechanism in HTTP can be used to store and transmit this session state.

5. Functions and characteristics of cookies

5-1. Create a cookie

If we set up a ‘stateful’ session first, we need:

1. When the server receives a request to establish the state (such as a login request), the server can return the response with a set-cookie header option that can represent the current client and the server (such as generating a unique sessionID).

2. The client receives set-cookie data and saves it (for example, the browser will automatically save it in a specified location).

3. Each request of the client will find the cookie related to the server of the current request from the specified location of cookie storage, and send the cookie request to the server.

In this way, a so-called ‘stateful’ session can be established between the client and the server.

const http = require('http'); const queryString = require('querystring'); const server = http.createServer(); server.on('request', (req, res) => { res.setHeader('Content-Type', 'text/html; charset=utf-8'); if (req.url == '/user') { let cookies = null; try { cookies = queryString.parse(req.headers.cookie, '; '); } catch (e) { res.statusCode = 401; Res.end (' no permissions '); } if (! cookies.user) { res.statusCode = 401; Res.end (' no permissions '); } else { res.end('Kobe Bryant and James'); } } if (if (req.method.toLowerCase() == 'post' && req.url == '/login') {) { let user = { id: 1, username: 'Kobe Bryant' }; // Set cookie res.setheader (' set-cookie ', 'user=${json.stringify (user)}'); . / / Set up multiple cookies / / res setHeader (' Set - cookies' [' a = 1 ', 'b = 2]). Res.end (' Authorization succeeded '); }}); server.listen(8888);Copy the code

Reference: developer.mozilla.org/zh-CN/docs/…

6. Use cookies in KOA to implement user session authentication

// C6-0-0
const Koa = require('koa');
const KoaRouter = require('koa-router');

const server = new Koa();
const router = new KoaRouter();

router.get('/user'.async (ctx, next) => {

    let user = null;

    try {
        user = ctx.cookies.get('user');
    } catch (e) {
        ctx.throw(401.'No permissions');
    }

    if(! user) { ctx.throw(401.'No permissions');
    } else {
        ctx.body = 'Kobe Bryant and James'; }}); router.post('/login'.async (ctx, next) => {
    let user = {
        id: 1.username: 'Kobe Bryant'
    };
    / / set the cookie
    ctx.cookies.set('user'.JSON.stringify(user));

    // Set multiple cookies
    // ctx.cookies.set('a', 1);
    // ctx.cookies.set('b', 2);

    ctx.body = 'Authorization successful';
})

server.use(router.routes());

server.listen(8888);
Copy the code

6-1. Cookie validity verification under Koa

To avoid Cookie tampering on the client side or during transmission, we can introduce some relevant validation schemes:

// C6-1-0
const Koa = require('koa');
const KoaRouter = require('koa-router');

const server = new Koa();
const router = new KoaRouter();

// The secret key used to generate the signature string
server.keys = ['KobeBryant'];

router.get('/user'.async (ctx, next) => {

    let user = null;

    try {
        user = ctx.cookies.get('user', {
          // Whether to verify the current cookie signature
          signed: true
        });
    } catch (e) {
        ctx.throw(401.'No permissions');
    }

    if(! user) { ctx.throw(401.'No permissions');
    } else {
        ctx.body = 'Kobe Bryant and James'; }}); router.post('/login'.async (ctx, next) => {
    let user = {
        id: 1.username: 'Kobe Bryant'
    };
    / / set the cookie
    ctx.cookies.set('user'.JSON.stringify(user), {
      // Whether to generate signatures at the same time
      signed: true
    });

    // Set multiple cookies
    // ctx.cookies.set('a', 1);
    // ctx.cookies.set('b', 2);

    ctx.body = 'Authorization successful';
})

server.use(router.routes());

server.listen(8888);
Copy the code

When sending cookies, the server generates a signed HASH string (as in this case, based on the key value of the current Cookie + secret key + SHA256) according to the specified algorithm and sends it to the client at the same time.

The client request will return the Cookie and the corresponding HASH to the server, and the back end will HASH the Cookie and the secret key saved by the client, and compare the calculated HASH with the HASH sent through the request. In this way, both the Cookie and the sent HASH will be different if they have been modified. To verify that the Cookie has been tampered with.

Protect the secret key!

6-2. Life cycle

Cookie can also be set to its valid time, corresponding to two values respectively:

  • Session: The default value. If the expiration time of a Cookie is set to session, then the browser closes and the session is automatically deleted.

  • Persistence, an expiration date specified by Expires or max-age

Setting Mode:

Expires=

Optional, set the current cookie expiration time, syntax:

Set-Cookie: <cookie-name>=<cookie-value>; Expires=<date>
Copy the code

Max-Age=

Optional, the number of seconds that need to pass before the cookie expires. A number of seconds of 0 or -1 will expire the cookie directly. Some older browsers (IE6, IE7, and IE8) do not support this property. For other browsers, if both Expires and max-age are present, then max-age takes precedence.

Set-Cookie: <cookie-name>=<cookie-value>; Max-Age=<non-zero-digit>
Copy the code

6-3. Scope

The Domain and Path identifiers define the * scope of cookies: * which urls are allowed to send cookies. Grammar:

Set-Cookie: <cookie-name>=<cookie-value>; Domain=<domain-value>
Set-Cookie: <cookie-name>=<cookie-value>; Path=<path-value>
Copy the code

6-4. Restrict access

There are two ways to ensure that cookies are sent securely and not accessed by unexpected actors or scripts: the Secure property and the HttpOnly property.

  • Marked asSecure 的 CookieShould only pass byHTTPSProtocol encrypted requests are sent to the server.
  • JavaScript Document.cookieAPI cannot access withHttpOnlyProperties of theCookie, limited toHTTPTransmission use.

Grammar:

Set-Cookie: <cookie-name>=<cookie-value>; Secure
Set-Cookie: <cookie-name>=<cookie-value>; HttpOnly
Copy the code

6-5. Use cases

Set-Cookie: <cookie-name>=<cookie-value>; Domain=<domain-value>; Secure; HttpOnly
Copy the code

Reference: developer.mozilla.org/zh-CN/docs/…