A small bug caused by URL coding thinking

scenario

Recently, when I was doing performance optimization, I found that an image that I wanted to load in advance did not hit the browser cache. It was clearly verified in the development environment, but when I went online, I found that two identical images were loaded, resulting in negative performance optimization.

So what happened to make that miss?

Let’s start with the status quo:

An ordinary VUE project, development environment, write a simple style, set up a background image

.xx {
  background-image: url(https://xx.cdn/xx.jpg?a=/nooo/nooo)}Copy the code

Then, in order to make the image load early, when it is ready to display, it can be displayed quickly (because the browser will notice the cache at this time) and optimize the user experience, so I wrote a simple JS in a front position to load the image early

const img = new Image()
img.src = 'https://xx.cdn/xx.jpg?a=/nooo/nooo'
Copy the code

It was fine when it was developed, but when it was packaged, the style was changed

.xx{background-image:url(https://xx.cdn/xx.jpg?a=%2Fnooo%2Fnooo)}
Copy the code

The core is that the query part of the image address A =/nooo/nooo is encoded as a=%2Fnooo%2Fnooo, the browser directly regarded these two as different images, resulting in no hit cache.

So, who did it?

It is estimated that when packaging, which loader or plug-in does encodeURIComponent to the value part of the query in the link

encodeURIComponent('/nooo/nooo') = = ='%2Fnooo%2Fnooo'
// encodeURI does not handle '/'
encodeURI('/nooo/nooo') = = ='/nooo/nooo'
Copy the code

Follow this train of thought to seek go down, discover, did not find at all!!

It seems that I still think too simple, can only go through a period of different (I am not so familiar with webpack related, so it took some time) investigation time, found that CSS in the last step of optimization, the link was encode. I use v4 version of @vue/ CLI-service, and the optimization logic in the code is as follows

// @vue/cli-service

const cssnanoOptions = {
  preset: ['default', {
    mergeLonghand: false.cssDeclarationSorter: false}}]if (rootOptions.productionSourceMap && sourceMap) {
  cssnanoOptions.map = { inline: false}}if (isProd) {
  webpackConfig
    .plugin('optimize-css')
      .use(require('@intervolga/optimize-cssnano-plugin'), [{
        sourceMap: rootOptions.productionSourceMap && sourceMap,
        cssnanoOptions
      }])
}
Copy the code

That is, the @intervolga/ optimization-cssnano plugin is called to optimize the CSS code, while this package calls cssnano to optimize it

// @intervolga/optimize-cssnano-plugin

const cssnano = require('cssnano');

const promise = postcss([cssnano(cssnanoOptions)]).
	...
Copy the code

Postcss-normalize-url: postcss-normalize-URL: postCSs-normalize-URL: postCSs-normalize-URL: postCSs-normalize-URL: postCSs-Normalize-URL: postCSs-Normalize-URL

// cssnano/packages/postcss-normalize-url

import normalize from 'normalize-url';

normalizedURL = normalize(url, options);
Copy the code

Now let’s see what’s going on inside the Normalize-URL. There is no encode code in it at all, and the operations most likely to cause encode are:

// normalize-url

const urlObj = new URL(urlString);

// Sort query parameters
if (opts.sortQueryParameters) {
  urlObj.searchParams.sort();
}
Copy the code

Does sort cause links to be encoded? Try it, and it does:

var a = new URL('https://xxx.com/add?a=/9&b=77')
console.log(a.search)
/ /? a=/9&b=77

a.searchParams.sort()
console.log(a.search)
/ /? a=%2F9&b=77
Copy the code

In fact, for URLSearchParams increase, delete, change and other operations, will lead to the parameters are encode, guess because these operations will let it decode, processing and then encode back.

So, what’s the solution?

Now that the reasons are known, the solutions are clear, mainly in two directions:

1. Change your code

In a strict sense, it is not very standard to write https://xx.cdn/xx.jpg?a=/nooo/nooo itself. You should encode all the parameters in query, that is, change all the places involved in the code to standard ones. Similar to https://xx.cdn/xx.jpg?a=%2Fnooo%2Fnooo

2. Change the packing configuration

In addition, after additional research, it can be found that CSsnano fixed this as a Bug in V5.0.11 and did not enable sortQueryParameters by default

5.0.11 (2021-11-16)

Bug fixes

• c38f14c3ce3d0: postcss-normalize-url: avoid changing parameter encoding

In addition, in fact, the latest V5 version of @vue/ CLI-service has used the new version of CSsnano, which normally does not have such a problem.

In this case, it seems understandable not to use sort, which can be turned off by modifying the configuration in version V4 of @vue/ CLI-service

// vue.config.js

module.exports = {
  chainWebpack: config= > {
    // Note that this is only required for formal environments
    if (process.env.NODE_ENV === 'production') {
      config.plugin('optimize-css').tap(([options]) = > {
        // Change the configuration directly, although I don't like the way hack is written
        options.cssnanoOptions.preset[1].normalizeUrl = {
          sortQueryParameters: false,}return [options]
      })
    }
  }
}
Copy the code

Over?

We seem to have found the cause, and we seem to have found the solution, and the problem itself is over.

However, what is the relationship between Encode and encodeURIComponent caused by the operation of URLSearchParams? Is it the same?

To answer this question, percent-encoding comes up

Percent-encoding

Percent-encoding, also known as URL encoding

Percent-encoding URL encoding The percent-encoding URL encoding is the percent-encoding of two hexadecimal characters. The detailed coding process is available at the WHATWG, in percent-encoded-bytes

URLSearchParams and encodeURIComponent are both percent-encoding, so their basic logic is the same, but their encoding ranges are different

According to whatWG, encodeURIComponent’s encoding range is Component Percent-encode set

The component percent-encode set is the userinfo percent-encode set and U+0024 ($) to U+0026 (&), inclusive, U+002B (+), and U+002C (,).

The encoding range involved in URLSearchParams is Application/X-www-form-urlencoded percent-encode set

The application/x-www-form-urlencoded percent-encode set is the component percent-encode set and U+0021 (!) , U+0027 (‘) to U+0029 RIGHT PARENTHESIS, inclusive, and U+007E (~).

It includes encodeURIComponent’s coding range, plus! , ‘, etc

There’s one more important difference: URLSearchParams encodes Spaces as plus signs (+), while encodeURIComponent encodes %20!

URLSearchParams objects will percent-encode anything in the application/x-www-form-urlencoded percent-encode set, and will encode U+0020 SPACE as U+002B (+).

With that in mind, go back to solution 1 (change your own code) and be careful, especially with the characters that happen to get stuck between URLSearchParams and encodeURIComponent:

var a = new URL('https://xxx.com/add?a=%209&b=! 77 ')
console.log(a.search)
/ /? a=%209&b=! 77

a.searchParams.sort()
console.log(a.search)
/ /? a=+9&b=%2177
Copy the code

In the example above, we thought it was coded (equivalent to encodeURIComponent) and got slapped in the face.

So the second solution seems more likely

The last

A brief summary is:

• @vue/cli-servicethev4Version inCSSThis parameter is used by default when optimizing compressionURLSearchParamsTo encode,v5The normal version does not have this problem, if you want to fix, you can directly closecssnanothesortQueryParameters
• URLSearchParamswithencodeURIComponentThe encoding logic is basically the same, but with a wider range and special treatment for whitespace