If you’re familiar with the Vary response header field, what does it do? When you examine a page with the PageSpeed tool, you often see suggestions like “Specify a Vary: accept-encoding header.” Why do you do this? This article, which documents some of my research on Vary, contains answers to these questions.
HTTP content negotiation
To understand the power of Vary, you need to understand HTTP’s content negotiation mechanism. Sometimes, the same URL can serve multiple different documents, which requires a mechanism between the server and client to select the most appropriate version, called content negotiation.
There are two ways to negotiate. One is that the server sends the list of available versions of the document to the client for the user to select. This can be done using the 300 Multiple Choices status code. This kind of scheme has many problems, first of all, a network round trip; Second, some versions of the same document on the server may be intended for clients with certain technical characteristics that ordinary users may not know about. For example, a server can typically output static resources as compressed and uncompressed versions. The compressed version is clearly intended for clients that support compression, but ordinary users are likely to choose the wrong version.
So HTTP content negotiation usually uses another scheme: the server automatically sends the most appropriate version based on certain fields in the request header sent by the client. There are two more types of request header fields that can be used for this mechanism: content-negotiation special fields (Accept fields) and other fields.
Take a look at the Accept field, as shown in the following table:
| Request header field | instructions | Response header field |
|---|---|---|
| Accept | Tells the server what media type to send | Content-Type |
| Accept-Language | Tells the server what language to send | Content-Language |
| Accept-Charset | Tells the server what character set to send | Content-Type |
| Accept-Encoding | Tell the server which compression method to use | Content-Encoding |
For example, the client sends the following request header:
Accept:*/* Accept-Encoding:gzip,deflate,sdch Accept-Language:zh-CN,en-US; Q = 0.8, en. Q = 0.6Copy the codeIndicates that it can accept resources of any MIME type; Support for gZIP, Deflate or SDCH compressed resources; The server supports zh-CN, EN-us, and EN, and the weight of zh-cn is the highest (q is 0-1, the highest is 1, the lowest is 0, and the default is 1). The server should preferentially return the version whose language is zh-CN.
A browser response header might look like this:
Content-Type: text/javascript
Content-Encoding: gzip
Copy the codeThe exact MIME type for this document is text/javascript; The document content is gzip compressed; The absence of a Content-language field in the response header usually means that the Language of the returned version is exactly the highest-weighted of the accept-language in the request header.
Sometimes, the above four Accept fields are not sufficient, such as the User-Agent field in the request header to output different content for a particular browser such as IE6. Similarly, cookies in the request header can be used by the server to output differentiated content.
Since there may be one or more intermediary entities (such as a cache server) between the client and the server, the most basic requirement of the cache service is to return the correct document to the user. If the server returns different content based on different user-agents, and the cache server caches the responses of IE6 users and returns them to users using other browsers, you will definitely have a problem.
So the HTTP protocol dictates that if the content provided by the server depends on a request header field “outside of the normal Accept negotiation field,” such as user-Agent, the Vary field must be included in the response header, and the Vary field must include user-Agent. Similarly, if the server used both user-Agent and Cookie fields in the request header to generate content, the Vary field in the response would look something like this:
Vary: User-Agent, Cookie
Copy the codeThat is, the Vary field is used to list a list of response fields that tells the cache server how to cache and filter the appropriate versions of documents for the same URL.
Buggy cache service
Look again at PageSpeed’s “Specify a Vary: The accept-encoding header is a special field for Content negotiation. The server only needs to add the content-Encoding field in the response header to specify the Content compression format. Or do not print content-encoding to indicate that the Content is uncompressed. The cache server should cache different Content for different content-encoding and return the most appropriate version based on the accept-Encoding field in the specific request.
But some bug-implemented cache servers ignore the Content-encoding in the response header, potentially returning a compressed version of the cache to clients that do not support compression. There are two ways to avoid this:
- Set the cache-Control field in the response header to private, telling intermediate entities not to Cache it;
- Add the Vary: accept-encoding response header to explicitly tell the cache server to cache different versions of the accept-Encoding field.
The second option is usually used to take advantage of the caching capabilities of intermediate entities.
For static resources such as CSS and JS, the server should always enable GZIP as long as the client supports it. Also, to prevent a buggy cache server from returning the wrong version to the user, you should print Vary: accept-encoding.
Nginx and SPDY
In general, the Web Server can do all of these things for us. For Nginx, the following configuration automatically adds Vary: accept-encoding to gZIP-enabled responses:
gzip_vary on;
Copy the codeCurl curl curl curl curl curl curl curl curl curl
jerry@www:~$ curl --head https://imququ.com/... /xx.js HTTP/1.1 200 OK Server: nginx Date: Tue, 31 Dec 2013 16:34:48 GMT Content-Type: application/x-javascript Content-Length: 66748 Last-Modified: Tue, 31 Dec 2013 14:30:52 GMT Connection: keep-alive Vary: Accept-Encoding ETag:"52c2d51c-104bc"
Expires: Fri, 29 Dec 2023 16:34:48 GMT
Cache-Control: max-age=315360000
Strict-Transport-Security: max-age=31536000
Accept-Ranges: bytes
Copy the codeAs you can see, the server prints “Vary: Accept-encoding” correctly, and everything is fine.
Using Chrome’s package capture tool, however, the response header looks like this:
HTTP/1.1 200 OK
cache-control: max-age=315360000
content-encoding: gzip
content-type: application/x-javascript
date: Tue, 31 Dec 2013 16:35:27 GMT
expires: Fri, 29 Dec 2023 16:35:27 GMT
last-modified: Tue, 31 Dec 2013 14:30:52 GMT
server: nginx
status: 200
strict-transport-security: max-age=31536000
version: HTTP/1.1
Copy the codeMy blog supports the SPDY/2 protocol. Accessing my blog with Chrome will result in SPDY, so the response headers look a bit unusual. For example, the field names are all lowercase. Additional status, Version, and other fields, which will be covered next time (note: see “Request/response headers in SPDY 3.1”). Amazingly, the Vary: accept-encoding in the response is missing, although nothing has changed on the server.
SPDY specifies that the client must support compression, which means that the SPDY server can directly enable compression without caring about the Accept-Encoding field in the request header. Nginx supports the SPDY/2 protocol.
User-agents are expected to support gzip and deflate compression. Regardless of the Accept-Encoding sent by the user-agent, the server may select gzip or deflate encoding at any time. [via]
Therefore, Vary: accept-Encoding is useless for spdy-enabled clients, and Nginx chose to remove it directly to save a bit of traffic. Curl or any other client that doesn’t support SPDY is still using HTTP, so the response header is normal.
The appropriateness of this approach for Nginx has been debated, and not all Spdy-enabled Web servers actually do this. For example, you can still see vary: accept-encoding even if you access the js file on the Google homepage using the SPDY protocol.
HTTP/1.1 200 OK
status: 200 OK
version: HTTP/1.1
age: 25762
alternate-protocol: 443:quic
cache-control: public, max-age=31536000
content-encoding: gzip
content-length: 154614
content-type: text/javascript; charset=UTF-8
date: Tue, 31 Dec 2013 23:23:51 GMT
expires: Wed, 31 Dec 2014 23:23:51 GMT
last-modified: Mon, 16 Dec 2013 21:54:35 GMT
server: sffe
vary: Accept-Encoding
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
Copy the codeIn addition, both Chrome and Firefox currently support the SPDY protocol, but PageSpeed Chrome and Firefox do not specifically address the SPDY protocol, so using them to test my blog will still prompt “Specify a Vary: Accept-encoding header “, which is a bit laughable. But The online version of PageSpeed has updated the rules, and an expanded version is expected soon. If you’re a neat freak, add “add_header vary accept-encoding;” to your Nginx configuration. Output the Vary response header manually.
PS: Vary in IE has a lot of holes, so be careful when using it. There are many articles in this section on the Internet, such as IE and Vary headers written by Hax in the early years, you can click on them.
Originally published in 2014-01-01 23:48:33.
I am the blogger of “JerryQu’s Little Station (imququ.com)”. My independent blog has been suspended for a few years now, and I will bring some useful content to the nuggets community in the near future, as well as write some new articles in nuggets, please follow me.