I wrote an article about the browser HTTP caching mechanism: the Browser HTTP caching mechanism
Unlike browsers, clients need a network framework to implement caching, and today we’ll take a look at Okhttp caching.
How to Use Cache (CacheControl)
OKhttp provides a selection of cache policies that are uniformly configured through CacheControl, which you can see through the constructor.
private CacheControl(boolean noCache, boolean noStore, int maxAgeSeconds, int sMaxAgeSeconds,
boolean isPrivate, boolean isPublic, boolean mustRevalidate, int maxStaleSeconds,
int minFreshSeconds, boolean onlyIfCached, boolean noTransform, boolean immutable,
@Nullable String headerValue) {}Copy the code| parameter | meaning |
|---|---|
| noCache | Doesn’t mean no caching. In effect, it means “revalidate with the server” before any cached response is used in each request |
| noStore | Nothing is cached, forced caching, comparison caching is not triggered |
| maxAgeSeconds | The cached contents will expire in XXX seconds. The cache expires |
| sMaxAgeSeconds | Distinct from maxAgeSeconds, this identifier shares the validity period of the cache |
| isPrivate | Only clients can cache, not the proxy layer |
| isPublic | Both client and proxy servers can be cached |
| mustRevalidate | Indicates that after a cache has expired, it cannot be used directly, but must be validated before it can be used, because there are some scenarios in which an expired cache can be used |
| maxStaleSeconds | Indicates that the client is willing to accept a response beyond its freshness life cycle, only beyond the additional time after maxAgeSeconds, the general cache effect time (maxAgeSeconds + maxStaleSeconds) |
| minFreshSeconds | Min-fresh requires the cache server to return cached data in min-Fresh time. For example, if a resource has been stored in the cache for 7s and “max-age=10”, then “7+1<10” will be fresh after 1s and therefore valid |
| onlyIfCached | Use only the data in the cache. If no data is cached, return 504 |
| immutable | immutable |
Example Command output:
Request request = new Request.Builder()
.url(httpUrl)
// Cache policy, enforce cache
.cacheControl(CacheControl.FORCE_CACHE)
.build();
Copy the codeWhen caching is enabled, we can see the following files in the cache directory:
Let’s take a look at some of the documents:
Journal file
59 b20da873a6f4c405e64ad3e29a24e9. 0 file
59 b20da873a6f4c405e64ad3e29a24e9. 1
You can see some of the header information about the network request, and some of the body information about the response, and that’s the cache file and that’s the main mechanism for caching;
CacheInterceptor
OKhttp cache implementation is done by CacheInterceptor, located in RetryAndFollowUpInterceptor and BridgeInterceptor, in real network connection, network requests before the interceptor.
@Override public Response intercept(Chain chain) throws IOException {
// 1. Get standby Response from cacheResponse cacheCandidate = cache ! =null
? cache.get(chain.request())
: null;
long now = System.currentTimeMillis();
// 2. Obtain the current network Request Request object and cache Response object according to the standby Response, cache policy, Request, etc
CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
Request networkRequest = strategy.networkRequest;
Response cacheResponse = strategy.cacheResponse;
//3. Track the current cache hit count and network request count for recording
if(cache ! =null) {
cache.trackResponse(strategy);
}
// 4. If the standby cache is unavailable, disable the standby cache
if(cacheCandidate ! =null && cacheResponse == null) {
closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
}
Build Response (Code 504) instead of using network request and caching result is invalid
// If we're forbidden from using the network and the cache is insufficient, fail.
if (networkRequest == null && cacheResponse == null) {
return new Response.Builder()
.request(chain.request())
.protocol(Protocol.HTTP_1_1)
.code(504)
.message("Unsatisfiable Request (only-if-cached)")
.body(Util.EMPTY_RESPONSE)
.sentRequestAtMillis(-1L)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
}
// 6. If the network is not allowed and the cache is available, the cache result is returned directly.
// If we don't need the network, we're done.
if (networkRequest == null) {
return cacheResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.build();
}
// 7. Start the follow-up interceptor to start the actual network request;
Response networkResponse = null;
try {
networkResponse = chain.proceed(networkRequest);
} finally {
// If we're crashing on I/O or otherwise, don't leak the cache body.
if (networkResponse == null&& cacheCandidate ! =null) { closeQuietly(cacheCandidate.body()); }}If the cloud resource has not changed, i.e. the code is 304, the cache result is directly used
if(cacheResponse ! =null) {
if (networkResponse.code() == HTTP_NOT_MODIFIED) {
Response response = cacheResponse.newBuilder()
.headers(combine(cacheResponse.headers(), networkResponse.headers()))
.sentRequestAtMillis(networkResponse.sentRequestAtMillis())
.receivedResponseAtMillis(networkResponse.receivedResponseAtMillis())
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
networkResponse.body().close();
// 9. The response entity has not changed and the cache needs to be updated to the latest response result
// Update the cache after combining headers but before stripping the
// Content-Encoding header (as performed by initContentStream()).
cache.trackConditionalCacheHit();
cache.update(cacheResponse, response);
return response;
} else{ closeQuietly(cacheResponse.body()); }}// 10. Build the final Response and store it in the cache if possible.
Response response = networkResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
if(cache ! =null) {
if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, networkRequest)) {
// Offer this request to the cache.
// 11. Write the network request to the cache and return a writable Response. Write the cache to the cache file at read time.
CacheRequest cacheRequest = cache.put(response);
return cacheWritingResponse(cacheRequest, response);
}
//12. Some of the following HTTP request methods are invalid when caching and need to delete cached data
if (HttpMethod.invalidatesCache(networkRequest.method())) {
try {
cache.remove(networkRequest);
} catch (IOException ignored) {
// The cache cannot be written.}}}return response;
}
public static boolean invalidatesCache(String method) {
return method.equals("POST")
|| method.equals("PATCH")
|| method.equals("PUT")
|| method.equals("DELETE")
|| method.equals("MOVE"); // WebDAV
}
Copy the codeThe overall distance is as follows:
- Get standby Response from cache first;
- Obtain the current network Request Request object and cache Response object according to the standby Response, cache policy and Request mentioned above.
- Track the current cache hit times and network request times for recording;
- If the standby cache is unavailable, disable the standby cache.
- Disallow network requests and cache results are invalid, build Response directly (Code 504);
- If the network is not allowed and the cache is available, the cache result is returned directly.
- Start executing the follow-up interceptor and start the actual network request;
- If the cloud resource has not changed, i.e. the code is 304, the cache result is directly used;
- The cloud response entity has not changed, so the cache needs to be updated to the latest response result.
- Build the final Response and store it in the cache if possible;
- Write the network request to the cache, and return a writable Response, and write the cache to the cache file at the time of reading;
- Some of the following HTTP request methods are invalid when caching and need to remove cached data
Cache (read)
In the first step above, we see that there is a member variable of InternalCache type Cache. Let’s look at how to get the Cache from the Cache.
@Nullable Response get(Request request) {
//1. Generate a unique key based on the Url;
String key = key(request.url());
DiskLruCache.Snapshot snapshot;
Entry entry;
try {
//2. Obtain the cache snapshot from the cache
snapshot = cache.get(key);
if (snapshot == null) {
return null; }}catch (IOException e) {
// Give up because the cache cannot be read.
return null;
}
METADATA (url, request method, protocol, certificate, return code, etc.);
try {
entry = new Entry(snapshot.getSource(ENTRY_METADATA));
} catch (IOException e) {
Util.closeQuietly(snapshot);
return null;
}
// 4. Build response based on cached data
Response response = entry.response(snapshot);
if(! entry.matches(request, response)) { Util.closeQuietly(response.body());return null;
}
return response;
}
Copy the codeThe cache data here uses a DiskLruCache. This time we will not go into the implementation of DiskLruCache.
- The cached key generates a unique identifier based on the URL (generated after md5 encryption of the URL and hexadecimal conversion)
- Obtain the cache Snapshot from cache(DiskLruCache). Here are the specific functions of the three classes
- An Entry is a data Entry that can be read from the cache or written to the cache.
- DiskLruCache.Snapshot Cache Snapshot, which provides the operation of reading and writing an entry.
- DiskLruCache.Editor A cache Editor used to edit data inside Entry, including write and read operations.
- Create an Entry from a Cache file, which is used to read headers from the Cache file, similar to the following:
* { * http://google.com/foo * GET * 2 * Accept-Language: fr-CA * Accept-Charset: Utf-8 * HTTP/1.1 200 OK * 3 * content-type: image/ PNG * Content-Length: 100 * cache-control: max-age=600 *} utF-8 * HTTP/1.1 200 OK * 3 * content-type: image/ PNG * Content-Length: 100 * cache-control: max-age=600 *}Copy the code - Build a Response object based on reading the cache data that has been read.
CacheStrategy
The required data has been read from the cache file above, but the decision to use caching, and how our configured caching policy takes effect, is governed by CacheStrategy. Let’s take a look at how you build a CacheStrategy
CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
Request networkRequest = strategy.networkRequest;
Response cacheResponse = strategy.cacheResponse;
Copy the codeThe Factory object is constructed to read cache-relevant Response header data from the cached Response.
public Factory(long nowMillis, Request request, Response cacheResponse) {
this.nowMillis = nowMillis;
this.request = request;
this.cacheResponse = cacheResponse;
if(cacheResponse ! =null) {
this.sentRequestMillis = cacheResponse.sentRequestAtMillis();
this.receivedResponseMillis = cacheResponse.receivedResponseAtMillis();
Headers headers = cacheResponse.headers();
for (int i = 0, size = headers.size(); i < size; i++) {
String fieldName = headers.name(i);
String value = headers.value(i);
if ("Date".equalsIgnoreCase(fieldName)) {
servedDate = HttpDate.parse(value);
servedDateString = value;
} else if ("Expires".equalsIgnoreCase(fieldName)) {
expires = HttpDate.parse(value);
} else if ("Last-Modified".equalsIgnoreCase(fieldName)) {
lastModified = HttpDate.parse(value);
lastModifiedString = value;
} else if ("ETag".equalsIgnoreCase(fieldName)) {
etag = value;
} else if ("Age".equalsIgnoreCase(fieldName)) {
ageSeconds = HttpHeaders.parseSeconds(value, -1); }}}}Copy the codeThe following logic is key to the effectiveness of caching mechanisms and caching policies.
public CacheStrategy get(a) {
CacheStrategy candidate = getCandidate();
If the network is not allowed and the cache is not available, return 504
if(candidate.networkRequest ! =null && request.cacheControl().onlyIfCached()) {
// We're forbidden from using the network and the cache is insufficient.
return new CacheStrategy(null.null);
}
return candidate;
}
Copy the code private CacheStrategy getCandidate(a) {
// There is no cached data, return directly
if (cacheResponse == null) {
return new CacheStrategy(request, null);
}
// HTTPS request, no handshake data, no cached data
// Drop the cached response if it's missing a required handshake.
if (request.isHttps() && cacheResponse.handshake() == null) {
return new CacheStrategy(request, null);
}
// Determine whether the current cache result is available, according to the response code and response header to determine which code can be cached can refer to the code
// If there is a "no-store" in the request header or response header, no cache is used
if(! isCacheable(cacheResponse, request)) {return new CacheStrategy(request, null);
}
If no cache is configured, or If "if-modified-since" or "if-none-match" exists in the request header, no cache is used.
CacheControl requestCaching = request.cacheControl();
if (requestCaching.noCache() || hasConditions(request)) {
return new CacheStrategy(request, null);
}
//
CacheControl responseCaching = cacheResponse.cacheControl();
// The current age of the cache response, which is how long the cache exists
long ageMillis = cacheResponseAge();
// How long does the cache expire
long freshMillis = computeFreshnessLifetime();
if(requestCaching.maxAgeSeconds() ! = -1) {
freshMillis = Math.min(freshMillis, SECONDS.toMillis(requestCaching.maxAgeSeconds()));
}
// Minimum cache, which requires the cache server to return the cached data in the min-fresh time
long minFreshMillis = 0;
if(requestCaching.minFreshSeconds() ! = -1) {
minFreshMillis = SECONDS.toMillis(requestCaching.minFreshSeconds());
}
// maxStaleMillis indicates that the client is willing to accept a response beyond its freshness lifecycle
long maxStaleMillis = 0;
if(! responseCaching.mustRevalidate() && requestCaching.maxStaleSeconds() ! = -1) {
maxStaleMillis = SECONDS.toMillis(requestCaching.maxStaleSeconds());
}
//ageMillis + minFreshMillis < freshMillis + maxStaleMillis
// Indicates that the current cache exists for less than the cache validity period and returns cached data.
if(! responseCaching.noCache() && ageMillis + minFreshMillis < freshMillis + maxStaleMillis) { Response.Builder builder = cacheResponse.newBuilder();if (ageMillis + minFreshMillis >= freshMillis) {
builder.addHeader("Warning"."110 HttpURLConnection \"Response is stale\"");
}
long oneDayMillis = 24 * 60 * 60 * 1000L;
if (ageMillis > oneDayMillis && isFreshnessLifetimeHeuristic()) {
builder.addHeader("Warning"."113 HttpURLConnection \"Heuristic expiration\"");
}
return new CacheStrategy(null, builder.build());
}
// Check whether the conditions of the current cached response header are met. If so, the cache Reponse is returned. Otherwise, the network request is normal.
String conditionName;
String conditionValue;
if(etag ! =null) {
conditionName = "If-None-Match";
conditionValue = etag;
} else if(lastModified ! =null) {
conditionName = "If-Modified-Since";
conditionValue = lastModifiedString;
} else if(servedDate ! =null) {
conditionName = "If-Modified-Since";
conditionValue = servedDateString;
} else {
return new CacheStrategy(request, null); // No condition! Make a regular request.
}
Headers.Builder conditionalRequestHeaders = request.headers().newBuilder();
Internal.instance.addLenient(conditionalRequestHeaders, conditionName, conditionValue);
Request conditionalRequest = request.newBuilder()
.headers(conditionalRequestHeaders.build())
.build();
return new CacheStrategy(conditionalRequest, cacheResponse);
}
Copy the codeAccording to the request and cache RePONse objects, the user configured cache policy and the cache policy information related to the response header, a series of judgments are made to determine whether to use cache data, and a cache policy object is constructed. For the cache header information involved, please refer to the description at the beginning of this chapter. It mainly involves the following logic
- “No-store” : If “no-store” exists in the request header or response header, no cache is used
- “No-cache” : No cache is used If no-cache is configured in the cache or If “if-modified-since” or “if-none-match” exists in the request header.
- Stale stale stale stale stale stale stale stale stale ageMillis + minFreshMillis < freshMillis + maxStaleMillis
- “If-none-match”, “ETag”, “if-modified-since” : When it comes to server cache, determine whether the server should return 304. See the browser cache logic for this piece of logic.
Cache handling
When Wang builds the cache policy object from the cache configuration policy, the subsequent logic goes back to CacheInterceptor:
5. Disallow network request and cache result is invalid, build Response directly (code 504); 6. If the network is not allowed and the cache is available, the cache result is directly returned. 7. Start executing subsequent interceptors and start actual network requests; 8. Judge if the cloud resource has not changed, i.e. the code is 304, and directly use the cache result; 9. The cloud response entity has not changed, so the cache needs to be updated as the latest response result; 10. Build the final Response and store it in the cache if possible. 11. Write the network request to the cache and return a writable Response. Write the cache to the cache file when reading. 12. When some of the HTTP request method cache is invalid (POST, a MOVE that PUT, PATCH, DELETE), need to DELETE the cached data.Copy the codeCache write logic
@Nullable CacheRequest put(Response response) {
String requestMethod = response.request().method();
// Some invalid cache data needs to be deleted
if (HttpMethod.invalidatesCache(response.request().method())) {
try {
remove(response.request());
} catch (IOException ignored) {
// The cache cannot be written.
}
return null;
}
// Non-get request, not written to cache
if(! requestMethod.equals("GET")) {
return null;
}
// If the mutable header contains an asterisk, it is not cached
if (HttpHeaders.hasVaryAll(response)) {
return null;
}
// The actual write cache operation
Entry entry = new Entry(response);
DiskLruCache.Editor editor = null;
try {
editor = cache.edit(key(response.request().url()));
if (editor == null) {
return null;
}
entry.writeTo(editor);
return new CacheRequestImpl(editor);
} catch (IOException e) {
abortQuietly(editor);
return null; }}Copy the code