preface
To be a good Android developer, you need a complete set ofThe knowledge systemHere, let us grow together into what we think ~.
In the first two articles, we have analyzed the core source code of View in detail — Android touch event transfer mechanism and The drawing process of Android View. Starting from this article, THE author will accompany you to in-depth analysis of most of the mainstream open source framework source code in Android. This will allow us to truly understand the ideas behind these excellent open source frameworks and really improve our internal skills. Currently, the sequence of analyses is as follows:
4. GreenDao Responsive Programming: 5. RxJava Memory Leak: 6. LeakCanary Dependency Injection: 7. ButterKnife 8. Dagger2 EventBus 9Copy the codeSummed up as a mind map, it looks like this:
This article will provide an in-depth analysis of Android’s three-party network library OKHttp source code. After reading OKHttp source code and a number of other excellent OKHttp source analysis articles, I found that if you understand the following three pieces, you can prove that you have a deep understanding of OKHttp.
- OKHttp request flow
- Network request cache processing
- The connection pool
First, a little Internet knowledge:
Some commonly used status codes
- 100 to 199: indicates that the request is received and processing continues
- 200 to 299: Indicates that the request is successfully received and understood
- 300 to 399: redirects. Further operations must be performed to complete the request
- 400 to 499: A client error occurs. The request has syntax errors or cannot be implemented
- 500 to 599: The server failed to implement the valid request
OKHttp request flow
A rough request flow diagram within OKHttp looks like this:
Here are the steps to make a Get request using OKHttp:
OKHttpClient client = new OKHttpClient (); Request Request = new request.builder ().url(url).build(); Response Response = client.newCall(request).execute();Copy the code1. Create an OKHttpClient
OkHttpClient client = new OkHttpClient();
public OkHttpClient() {
this(new Builder());
}
OkHttpClient(Builder builder) {
....
}
Copy the codeAs you can see, OkHttpClient uses Builder mode. The parameters in Builder are as follows:
public static final class Builder { Dispatcher dispatcher; // @nullable Proxy Proxy; List<Protocol> protocols; List<ConnectionSpec> connectionSpecs; Final List<Interceptor> interceptors = new ArrayList<>(); Final List<Interceptor> networkInterceptors = new ArrayList<>(); EventListener.Factory eventListenerFactory; ProxySelector proxySelector; CookieJar cookieJar; @Nullable Cache cache; @Nullable InternalCache internalCache; SocketFactory SocketFactory; @Nullable SSLSocketFactory sslSocketFactory; // Secure Socket factory, used in HTTPS @nullable CertificateChainCleaner; // Verify that the response certificate applies to the host name of the HTTPS request connection. HostnameVerifier hostnameVerifier; // Verify that the response certificate applies to the host name of the HTTPS request connection. CertificatePinner certificatePinner; // Certificate locking, using a CertificatePinner to constrain which certification authorities are trusted. Authenticator proxyAuthenticator; // Proxy authentication Authenticator Authenticator; ConnectionPool ConnectionPool; // Connection pool Dns Dns; boolean followSslRedirects; // Redirect Boolean uploader; // Local redirection Boolean retryOnConnectionFailure; // Retry connection failure int callTimeout; int connectTimeout; int readTimeout; int writeTimeout; int pingInterval; Public Builder() {dispatcher = new dispatcher (); protocols = DEFAULT_PROTOCOLS; connectionSpecs = DEFAULT_CONNECTION_SPECS; . } // this = okHttpClient.dispatcher; this = okHttpClient.dispatcher; this.proxy = okHttpClient.proxy; this.protocols = okHttpClient.protocols; this.connectionSpecs = okHttpClient.connectionSpecs; this.interceptors.addAll(okHttpClient.interceptors); this.networkInterceptors.addAll(okHttpClient.networkInterceptors); . }Copy the code2. Synchronize the request process
Response response = client.newCall(request).execute(); /** * Prepares the {@code request} to be executed at some point in the future. */ @Override public Call newCall(Request request) { return RealCall.newRealCall(this, request, false /* for web socket */); Static RealCall newRealCall(OkHttpClient client, Request originalRequest, boolean forWebSocket) { // Safely publish the Call instance to the EventListener. RealCall call = new RealCall(client, originalRequest, forWebSocket); call.eventListener = client.eventListenerFactory().create(call); return call; } @override public Response execute() throws IOException {synchronized (this) {// Only one Call can be executed if (executed) throw new IllegalStateException("Already Executed"); executed = true; } captureCallStackTrace(); timeout.enter(); eventListener.callStart(this); Client.dispatcher ().executed(this); / / through a series of interceptor request processing and Response processing to get the final returns the Response result = getResponseWithInterceptorChain (); if (result == null) throw new IOException("Canceled"); return result; } catch (IOException e) { e = timeoutExit(e); eventListener.callFailed(this, e); throw e; } finally {// Notify the dispatcher that it has finished executing client.dispatcher().finished(this); } } Response getResponseWithInterceptorChain() throws IOException { // Build a full stack of interceptors. List<Interceptor> interceptors = new ArrayList<>(); // Interceptors set when configuring OkHttpClient; interceptors.addAll(client.interceptors()); / / is responsible for the failure retry and redirect interceptors. Add (retryAndFollowUpInterceptor); Add (new BridgeInterceptor(client.cookiejar ()))); Add (new CacheInterceptor(client.internalCache())); Interceptors. add(new ConnectInterceptor(client)); if (! ForWebSocket) {/ / configuration Settings when OkHttpClient networkInterceptors interceptors. AddAll (client.net workInterceptors ()); } // Interceptors. Add (new CallServerInterceptor(forWebSocket)); Interceptor.Chain chain = new RealInterceptorChain(interceptors, null, null, null, 0, originalRequest, this, eventListener, client.connectTimeoutMillis(), client.readTimeoutMillis(), client.writeTimeoutMillis()); Return chain.proceed(originalRequest); } // StreamAllocation object, which acts as a management class. It maintains the relationship between server connections, concurrent streams, and requests. It also initializes a Socket connection object to get input/output streams. public Response proceed(Request request, StreamAllocation streamAllocation, HttpCodec httpCodec, RealConnection connection) throws IOException { ... Call the next interceptor in the chain. // Instantiate the RealIterceptorChain object corresponding to the next interceptor RealInterceptorChain next = new RealInterceptorChain(interceptors, streamAllocation, httpCodec, connection, index + 1, request, call, eventListener, connectTimeout, readTimeout, writeTimeout); Interceptor = interceptors.get(index); Response Response = interceptor.intercept(next); Response = interceptor.intercept(next); Response = interceptor.intercept(next); . return response; }Copy the code3. Asynchronous request process
Request request = new Request.Builder() .url("http://publicobject.com/helloworld.txt") .build(); client.newCall(request).enqueue(new Callback() { @Override public void onFailure(Call call, IOException e) { e.printStackTrace(); } @Override public void onResponse(Call call, Response response) throws IOException { ... } void enqueue(AsyncCall call) { synchronized (this) { readyAsyncCalls.add(call); } promoteAndExecute(); } private final Deque<AsyncCall> readyAsyncCalls = new ArrayDeque<>(); Private final Deque<AsyncCall> runningAsyncCalls = new ArrayDeque<>(); private final Deque<AsyncCall> runningAsyncCalls = new ArrayDeque<>(); Private final Deque<RealCall> runningSyncCalls = new ArrayDeque<>(); // Promotes eligible calls from {@link #readyAsyncCalls} to {@link #runningAsyncCalls} and runs // them on the executor service. Must not be called with synchronization because executing calls // can call into user code. private boolean promoteAndExecute() { assert (! Thread.holdsLock(this)); List<AsyncCall> executableCalls = new ArrayList<>(); boolean isRunning; synchronized (this) { for (Iterator<AsyncCall> i = readyAsyncCalls.iterator(); i.hasNext(); ) { AsyncCall asyncCall = i.next(); // If the number of runningAsyncCalls is insufficient and the number of hosts occupied by calls is less than the maximum number, the call is added to runningAsyncCalls. Otherwise, add call to readyAsyncCalls. if (runningAsyncCalls.size() >= maxRequests) break; // Max capacity. if (runningCallsForHost(asyncCall) >= maxRequestsPerHost) continue; // Host max capacity. i.remove(); executableCalls.add(asyncCall); runningAsyncCalls.add(asyncCall); } isRunning = runningCallsCount() > 0; } for (int i = 0, size = executableCalls.size(); i < size; i++) { AsyncCall asyncCall = executableCalls.get(i); asyncCall.executeOn(executorService()); } return isRunning; }Copy the codeFinally, we’ll look at the code for AsynCall.
final class AsyncCall extends NamedRunnable { private final Callback responseCallback; AsyncCall(Callback responseCallback) { super("OkHttp %s", redactedUrl()); this.responseCallback = responseCallback; } String host() { return originalRequest.url().host(); } Request request() { return originalRequest; } RealCall get() { return RealCall.this; } /** * Attempt to enqueue this async call on {@code executorService}. This will attempt to clean up * if the executor has been shut down by reporting the call as failed. */ void executeOn(ExecutorService executorService) { assert (! Thread.holdsLock(client.dispatcher())); boolean success = false; try { executorService.execute(this); success = true; } catch (RejectedExecutionException e) { InterruptedIOException ioException = new InterruptedIOException("executor rejected"); ioException.initCause(e); eventListener.callFailed(RealCall.this, ioException); responseCallback.onFailure(RealCall.this, ioException); } finally { if (! success) { client.dispatcher().finished(this); // This call is no longer running! } } } @Override protected void execute() { boolean signalledCallback = false; timeout.enter(); Try {/ / same as synchronous implementation, finally will be called to Response the Response = getResponseWithInterceptorChain (); if (retryAndFollowUpInterceptor.isCanceled()) { signalledCallback = true; responseCallback.onFailure(RealCall.this, new IOException("Canceled")); } else { signalledCallback = true; responseCallback.onResponse(RealCall.this, response); } } catch (IOException e) { e = timeoutExit(e); if (signalledCallback) { // Do not signal the callback twice! Platform.get().log(INFO, "Callback failure for " + toLoggableString(), e); } else { eventListener.callFailed(RealCall.this, e); responseCallback.onFailure(RealCall.this, e); } } finally { client.dispatcher().finished(this); }}}Copy the codeFrom the source code, can know intercept chain processing OKHttp help us default did five steps to intercept processing, including RetryAndFollowUpInterceptor, BridgeInterceptor, CallServerInterceptor internal source code is very concise and easy to understand, Without further ado, I’ll cover the two core parts of OKHttp: caching and connection handling (connection pooling).
CacheInterceptor for network request caching
@override public Response Intercept (Chain Chain) throws IOException {// Obtain the Response cached in the cache according to request cacheCandidate = cache ! = null ? cache.get(chain.request()) : null; long now = System.currentTimeMillis(); // Request to determine the cache policy, whether to use the network, CacheStrategy Strategy = new cacheStrategy.factory (now, chain-.request (), cacheCandidate).get(); Request networkRequest = strategy.networkRequest; Response cacheResponse = strategy.cacheResponse; if (cache ! = null) { cache.trackResponse(strategy); } if (cacheCandidate ! = null && cacheResponse == null) { closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it. } // If we're forbidden from using the network and the cache is insufficient, fail. if (networkRequest == null && cacheResponse == null) { return new Response.Builder() .request(chain.request()) .protocol(Protocol.HTTP_1_1) .code(504) .message("Unsatisfiable Request (only-if-cached)") .body(Util.EMPTY_RESPONSE) .sentRequestAtMillis(-1L) .receivedResponseAtMillis(System.currentTimeMillis()) .build(); } // If we don't need the network, we're done. if (networkRequest == null) { return cacheResponse.newBuilder() .cacheResponse(stripBody(cacheResponse)) .build(); } Response networkResponse = null; Try {// call the next interceptor, decide to get response from the network networkResponse = chain.proceed(networkRequest); } finally { // If we're crashing on I/O or otherwise, don't leak the cache body. if (networkResponse == null && cacheCandidate ! = null) { closeQuietly(cacheCandidate.body()); }} If we have a cache response too, then we're doing a conditional get. So compare it to the networkResponse and decide whether to update the cached cacheResponse if (cacheResponse! = null) { if (networkResponse.code() == HTTP_NOT_MODIFIED) { Response response = cacheResponse.newBuilder() .headers(combine(cacheResponse.headers(), networkResponse.headers())) .sentRequestAtMillis(networkResponse.sentRequestAtMillis()) .receivedResponseAtMillis(networkResponse.receivedResponseAtMillis()) .cacheResponse(stripBody(cacheResponse)) .networkResponse(stripBody(networkResponse)) .build(); networkResponse.body().close(); // Update the cache after combining headers but before stripping the // Content-Encoding header (as performed by initContentStream()). cache.trackConditionalCacheHit(); cache.update(cacheResponse, response); return response; } else { closeQuietly(cacheResponse.body()); } } Response response = networkResponse.newBuilder() .cacheResponse(stripBody(cacheResponse)) .networkResponse(stripBody(networkResponse)) .build(); if (cache ! = null) { if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, NetworkRequest)) {// Offer this request to the cache. // Cache an uncached Response CacheRequest CacheRequest = cache.put(response); return cacheWritingResponse(cacheRequest, response); } if (HttpMethod.invalidatesCache(networkRequest.method())) { try { cache.remove(networkRequest); } catch (IOException ignored) { // The cache cannot be written. } } } return response; }Copy the codeThe cache interceptor determines whether a cache is available based on the information about the request and the cached response. If one is available, it returns the cache to the user, otherwise it continues to retrieve the response from the server in chain of responsibility mode. When the response is retrieved, it is cached to disk.
ConnectInterceptor’s connection pool
@Override public Response intercept(Chain chain) throws IOException { RealInterceptorChain realChain = (RealInterceptorChain) chain; Request request = realChain.request(); StreamAllocation streamAllocation = realChain.streamAllocation(); // We need the network to satisfy this request. Possibly for validating a conditional GET. boolean doExtensiveHealthChecks = ! request.method().equals("GET"); // HttpCodec is an abstraction of HTTP protocol operations. There are two implementations: Http1Codec and Http2Codec, which, as the name implies, correspond to HTTP/1.1 and HTTP/2 versions, respectively. In this method the internal implementation of connection pool in the reuse treatment HttpCodec HttpCodec = streamAllocation. NewStream (client, chain, doExtensiveHealthChecks); RealConnection connection = streamAllocation.connection(); return realChain.proceed(request, streamAllocation, httpCodec, connection); } // Returns a connection to host a new stream. This // prefers the existing connection if it exists, // then the pool, Finally building a new connection. // When calling streamAllocation's newStream() method, The findConnection() method in StreamAllocation is finally reached after a series of // judgments. Private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout, int pingIntervalMillis, boolean connectionRetryEnabled) throws IOException { ... // Attempt to use an already-allocated connection. We need to be careful here because our // already-allocated Connection may have been restricted from creating new streams. Connections that have already been allocated may have been restricted from creating new streams releasedConnection = this.connection; ToClose = releaseIfNoNewStreams(); toClose = releaseIfNoNewStreams(); toClose = releaseIfNoNewStreams(); if (this.connection ! = null) { // We had an already-allocated connection and it's good. result = this.connection; releasedConnection = null; } if (! reportedAcquired) { // If the connection was never reported acquired, don't report it as released! // If the connection is never marked as acquired, do not mark it as published. ReportedAcquired () = NULL; } if (result == null) {// Attempt to get a connection from the pool Internal.instance.get(connectionPool, address, this, null); if (connection ! = null) { foundPooledConnection = true; result = connection; } else { selectedRoute = route; }}} // closeQuietly(toClose); if (releasedConnection ! = null) { eventListener.connectionReleased(call, releasedConnection); } if (foundPooledConnection) { eventListener.connectionAcquired(call, result); } if (result ! Allocated tokens = "allocated tokens" = "allocated tokens" = "allocated tokens" = "allocated tokens" = "allocated tokens"; Return result; } // If we need a route selection, make one. This is a blocking operation. boolean newRouteSelection = false; if (selectedRoute == null && (routeSelection == null || ! routeSelection.hasNext())) { newRouteSelection = true; routeSelection = routeSelector.next(); } synchronized (connectionPool) { if (canceled) throw new IOException("Canceled"); if (newRouteSelection) { // Now that we have a set of IP addresses, Make another attempt at getting a connection from // The pool. This could match due to connection coalescing List<Route> routes = routeselection.getall (); for (int i = 0, size = routes.size(); i < size; i++) { Route route = routes.get(i); Get (connectionPool, address, this, route); if (connection ! = null) { foundPooledConnection = true; result = connection; this.route = route; break; } } } if (! foundPooledConnection) { if (selectedRoute == null) { selectedRoute = routeSelection.next(); } // Create a connection and assign it to this allocation immediately. This makes it possible // for an asynchronous Cancel () to interrupt the handshake we're about to do. // Create a new connection if there is no one in the connection pool and assign it, That way we can end the handshake before it happens. refusedStreamCount = 0; result = new RealConnection(connectionPool, selectedRoute); acquire(result, false); } } // If we found a pooled connection on the 2nd time around, We're done. If (foundPooledConnection) {// If we found a pool connection the second time, Then we will return to eventListener. Its connectionAcquired (call, result); return result; } // Do TCP + TLS handshakes. This is a blocking operation. readTimeout, writeTimeout, pingIntervalMillis, connectionRetryEnabled, call, eventListener); routeDatabase().connected(result.route()); Socket socket = null; synchronized (connectionPool) { reportedAcquired = true; // Pool the connection. // Add the connection to the Pool internal.instance. // If another multiplexed connection to the same address was created concurrently, Then // release this connection and acquire that one. Release and obtain the connection if the connection (result) isMultiplexed ()) {socket = Internal. Instance. Deduplicate (connectionPool, address, this); result = connection; } } closeQuietly(socket); eventListener.connectionAcquired(call, result); return result; }Copy the codeFrom the above source code analysis:
- Determine whether the current connection is available: whether the stream has been closed and is restricted from creating new streams;
- If the current connection is not available, get a connection from the connection pool;
- There are no available connections in the connection pool, create a new connection, shake hands, and then put it into the connection pool.
The Internal get() method is used to retrieve a connection from the connection pool. Internal has a static instance that is initialized in the OkHttpClient static code quick. We call the connection pool get() method in the Internal get() to get a connection. Also, one of the benefits of connection reuse is to eliminate the need for a TCP/TLS handshake. Since establishing a connection itself takes some time, it can be reused to improve the efficiency of our network access.
Next, let’s take a closer look at how ConnectionPool implements connection management.
OkHttp cache management is divided into two steps. On the one hand, when we create a new connection, we put it in the cache. On the other hand, we have to clean up the cache. In ConnectionPool, when we cache a connection to the ConnectionPool, we simply call the add() method of the two-end queue to add it to the two-end queue, and the operation of cleaning the connection cache is left to the thread pool to perform periodically.
private final Deque<RealConnection> connections = new ArrayDeque<>(); void put(RealConnection connection) { assert (Thread.holdsLock(this)); if (! cleanupRunning) { cleanupRunning = true; Executor. execute(cleanupRunnable) using a thread pool; } // Insert the new connection into the double-ended queue connections.add(connection); } private final Runnable cleanupRunnable = new Runnable() {@override public void run() {while (true) {// Internal call Cleanup () method to cleanup invalid connections long waitNanos = cleanup(system.nanotime ()); if (waitNanos == -1) return; if (waitNanos > 0) { long waitMillis = waitNanos / 1000000L; waitNanos -= (waitMillis * 1000000L); synchronized (ConnectionPool.this) { try { ConnectionPool.this.wait(waitMillis, (int) waitNanos); } catch (InterruptedException ignored) { } } } } }; long cleanup(long now) { int inUseConnectionCount = 0; int idleConnectionCount = 0; RealConnection longestIdleConnection = null; long longestIdleDurationNs = Long.MIN_VALUE; // Find either a connection to evict, Or the time that the next eviction is due. Synchronized (this) {// Traverse all connections for (Iterator<RealConnection> I = connections.iterator(); i.hasNext(); ) { RealConnection connection = i.next(); / / If the connection is in use, keep searching. / / to iterate through all the connection of the If (pruneAndGetAllocationCount (connection, now) > 0) { inUseConnectionCount++; continue; } idleConnectionCount++; // If the connection is ready to be evicted, we're done. Long idleDurationNs = now-Connection. idleAtNanos; long idleDurationNs = now-connection. idleAtNanos; if (idleDurationNs > longestIdleDurationNs) { longestIdleDurationNs = idleDurationNs; longestIdleConnection = connection; }} // maxIdleConnections specifies the maximum number of connections allowed to be idle, and keepAliveDurationNs specifies the maximum duration of connections allowed to live. // By default, the maximum number of idle connections is 5 and the keepalive duration is 5 minutes. if (longestIdleDurationNs >= this.keepAliveDurationNs || idleConnectionCount > this.maxIdleConnections) { // We've found a connection to evict. Remove it from the list, Then close it below (outside // of the synchronized block). // The connection length exceeds the maximum active time or the number of idle connections exceeds the maximum allowed range, Remove connections.remove(longestIdleConnection); } else if (idleConnectionCount > 0) {// A connection will be ready to evict soon. Not yet) return keepAliveDurationNs - longestIdleDurationNs; } else if (inUseConnectionCount > 0) { // All connections are in use. It'll be at least the keep alive duration 'til we Return keepAliveDurationNs; run again. // All connections are in use, 5 minutes later to clean up keepAliveDurationNs; } else {// No connections, idle or in use. return -1; }}Copy the codeAs you can see from the above source code analysis, connections in the cache are first traversed to find the one that has been idle for the longest time, and then parameters such as the idle time of the connection and the maximum number of allowed connections are used to determine whether the connection should be cleaned up. If the connection that has been idle for the longest time still needs to be cleaned up, the time difference will be returned, and the connection pool will be cleaned up again after this time.
Four,
After analyzing the inner workings of OKHttp above, I’m sure you already have a good understanding of OKHttp. At first, we will request to initialize an instance of the Call and then execute it the execute () method or the enqueue () method, internal finally will be executed to getResponseWithInterceptorChain () method, this method through the interceptor chain of responsibility, A response is obtained and delivered to the user through the following interception processes: user-defined normal interceptor, retry interceptor, bridge interceptor, cache interceptor, connection interceptor, user-defined network interceptor, and access server interceptor. In addition to the internal request flow of OKHttp, caching and connection are also two important points. I believe that after the explanation of this article, readers have their own understanding of the three parts of the key content. We’ll look into the source code for Retrofit, OKHttp’s encapsulation framework
Reference links:
OKHttp V3.12.0 source code
1. Android Advanced Light
2. OKHttp source code analysis
3, Andriod network framework OkHttp source code analysis