preface

The first two articles explained Volley, HttpURLConnection, and today is the analysis of OKHttp. After the analysis, OKIO and Retrofit will be analyzed. Through this series of analysis, we try to have a sufficient understanding of the implementation of Android network library. Of course, the analysis here is completely divorced from the specific practice of the project, so there will be lack of some details, which will be the next step in the source code analysis, from the project application and practice optimization as a starting point.

Based on using

  • Create the OKHttpClient and construct the request
OkHttpClient client = new OkHttpClient.Builder()
        .readTimeout(30, TimeUnit.SECONDS)
        .build();

Request request = new Request.Builder()
        .header("User-Agent", "OkHttp Headers.java")
        .url("http://www.baidu.com")
        .build();
Copy the code
  • Making an asynchronous request
client.newCall(request).enqueue(new Callback() {
    @Override
    public void onFailure(Call call, IOException e) {
    }

    @Override
    public void onResponse(Call call, okhttp3.Response response) throws IOException {
    }
});
Copy the code
  • Initiating a synchronization request
okhttp3.Response response = client.newCall(request).execute();
Copy the code

The OkHttp generator creates an OKHttpClient according to our own configuration, and then constructs a request, setting the header, URL, and body of the request. OkHttp provides both synchronous and asynchronous request execution modes. Here you can choose an appropriate way according to your needs.

Implementation analysis

In accordance with the Android Weekly wheel writing style, the basic use as an introduction to help us quickly cut into the implementation process of the framework, quickly clarify the whole call link, understand the implementation of the framework, here we continue this way. Step by step analysis for the above use process.

  • newCall
public Call newCall(Request request) {
  return RealCall.newRealCall(this, request, false /* for web socket */);
}
Copy the code
static RealCall newRealCall(OkHttpClient client, Request originalRequest, boolean forWebSocket) {
  RealCall call = new RealCall(client, originalRequest, forWebSocket);
  call.eventListener = client.eventListenerFactory().create(call);
  return call;
}
Copy the code

Based on the request created, a RealCall instance is constructed and event listeners are set for the RealCall object. In EventListener, a number of functions are defined to monitor the entire life cycle of a network request, including DNS lookup start, DNS lookup end, connection establishment start, connection establishment failure, and so on.

  • enqueue

After the connection is established, the enqueue method is called in an asynchronous request.

public void enqueue(Callback responseCallback) {
  synchronized (this) {
    if (executed) throw new IllegalStateException("Already Executed");
    executed = true;
  }
  captureCallStackTrace();
  eventListener.callStart(this);
  client.dispatcher().enqueue(new AsyncCall(responseCallback));
}
Copy the code

In this method, the enqueue method of OkHttpClient’s Dispatcher is called. When creating OkHttpClient, a Dispatcher is created by default if the developer has not specified one. The analysis follows the default implementation code.

synchronized void enqueue(AsyncCall call) { if (runningAsyncCalls.size() < maxRequests && runningCallsForHost(call) < maxRequestsPerHost) { runningAsyncCalls.add(call); executorService().execute(call); } else { readyAsyncCalls.add(call); }}Copy the code

After joining Dispatchr, it determines whether the maximum number of requests has been exceeded, whether the number of requests under a single host share has been exceeded, and if so, it is added to the ready execution queue. If there are too many requests, place them in a ready queue. The data structure of ArrayDeque is explained at the end of the article. Network request execution is realized by thread pool. The creation and execution of thread pool will be analyzed in detail at the end of this paper.

The data structures used to manage requests in the Dispatcher are asynchronous request queues in progress, asynchronous request queues in preparation, and synchronous request queues in progress.

Deque<AsyncCall> runningAsyncCalls = new ArrayDeque<>();
Deque<AsyncCall> readyAsyncCalls = new ArrayDeque<>();
Deque<RealCall> runningSyncCalls = new ArrayDeque<>();
Copy the code
  • execute

The specific execution of the request is performed in the AsyncCall execute method.

final class AsyncCall extends NamedRunnable { @Override protected void execute() { boolean signalledCallback = false; try { Response response = getResponseWithInterceptorChain(); . } catch (IOException e) { ...... } finally { client.dispatcher().finished(this); }}}Copy the code

This is where you get the response first, and when you get the response, you might get some exception, you catch the exception, you call back some of the callback functions of the event listener. At the heart of the call is to get request response result getResponseWithInterceptorChain through layers of execution of the chain of responsibility for the request of the final results.

  • getResponseWithInterceptorChain
Response getResponseWithInterceptorChain() throws IOException { List<Interceptor> interceptors = new ArrayList<>(); interceptors.addAll(client.interceptors()); interceptors.add(retryAndFollowUpInterceptor); interceptors.add(new BridgeInterceptor(client.cookieJar())); interceptors.add(new CacheInterceptor(client.internalCache())); interceptors.add(new ConnectInterceptor(client)); if (! forWebSocket) { interceptors.addAll(client.networkInterceptors()); } interceptors.add(new CallServerInterceptor(forWebSocket)); Interceptor.Chain chain = new RealInterceptorChain(interceptors, null, null, null, 0, originalRequest, this, eventListener, client.connectTimeoutMillis(), client.readTimeoutMillis(), client.writeTimeoutMillis()); return chain.proceed(originalRequest); }Copy the code

Build a list of interceptors, adding user-set interceptors, the framework’s own cache, network connections, and so on. Then build an instance object based on this list of interceptors and call its processed method.

  • Chain of Responsibility execution (PROCEED)

The proceed method is at the heart of the network request execution

public Response proceed(Request request, StreamAllocation streamAllocation, HttpCodec httpCodec, RealConnection connection) throws IOException { if (index >= interceptors.size()) throw new AssertionError(); calls++; // If there is already an existing stream, the incoming request can use it. = null && ! this.connection.supportsUrl(request.url())) { throw new IllegalStateException("network interceptor " + interceptors.get(index - 1) + " must retain the same host and port"); } if (this.httpCodec ! = null && calls > 1) { throw new IllegalStateException("network interceptor " + interceptors.get(index - 1) + " must call proceed() exactly once"); } RealInterceptorChain next = new RealInterceptorChain(Interceptors, streamAllocation, httpCodec, connection, index + 1, request, call, eventListener, connectTimeout, readTimeout, writeTimeout); Interceptor interceptor = interceptors.get(index); Response response = interceptor.intercept(next); // Verify that the interceptor calls the chain's proceed method. if (httpCodec ! = null && index + 1 < interceptors.size() && next.calls ! = 1) { throw new IllegalStateException("network interceptor " + interceptor + " must call proceed() exactly once"); } // Check if the response is null, If (response == null) {throw new NullPointerException("interceptor "+ interceptor +" returned null"); } // Check whether the response body is empty, If (Response.body () == null) {throw new IllegalStateException("interceptor "+ interceptor +" returned A response with no body"); } return response; }Copy the code

The chain of responsibility handler roughly evaluates some state, takes an interceptor from it, constructs an instance of the interceptor chain, and executes the interceptor’s interceptor method, in which it also calls the proceed method of the newly created RealChain method. In this recursive way, the data is wrapped up in layers and finally thrown back. Each of these interceptors plays a key role in completing the entire network request. The next step is a level by level analysis, starting with the first interceptor.

RetryAndFollowUpInterceptor

The interceptor can recover from failures and necessary redirects. How many redirects and authorizations need to be tried, Chrome will follow 21 redirects, Firefox, Curl, wget will follow 20 redirects, Safari will follow 16 redirects, Http1.0 recommended 5 redirects, and here it is 20 redirects.

Retry is done through a While True loop, where StreamAllocation is established, then a new instance of the interceptor chain is created, and its processed method is called, waiting for the result to return. The interceptor is the outermost layer, and any response returned by subsequent interceptors will eventually be returned to this layer for processing.

@Override public Response intercept(Chain chain) throws IOException { Request request = chain.request(); RealInterceptorChain realChain = (RealInterceptorChain) chain; Call call = realChain.call(); EventListener eventListener = realChain.eventListener(); StreamAllocation streamAllocation = new StreamAllocation(client.connectionPool(), createAddress(request.url()), call, eventListener, callStackTrace); this.streamAllocation = streamAllocation; int followUpCount = 0; Response priorResponse = null; while (true) { ..... Response response; boolean releaseConnection = true; try { response = realChain.proceed(request, streamAllocation, null, null); releaseConnection = false; } catch (RouteException e) {// Recover if (! recover(e.getLastConnectException(), streamAllocation, false, request)) { throw e.getLastConnectException(); } releaseConnection = false; continue; } catch (IOException e) { boolean requestSendStarted = ! (e instanceof ConnectionShutdownException); if (! recover(e, streamAllocation, requestSendStarted, request)) throw e; releaseConnection = false; continue; } finally { if (releaseConnection) { streamAllocation.streamFailed(null); streamAllocation.release(); } } Request followUp = followUpRequest(response, streamAllocation.route()); if (followUp == null) { if (! forWebSocket) { streamAllocation.release(); } return response; } closeQuietly(response.body()); If (++followUpCount > MAX_FOLLOW_UPS) {streamAllocation. Release (); throw new ProtocolException("Too many follow-up requests: " + followUpCount); } if (followUp.body() instanceof UnrepeatableRequestBody) { streamAllocation.release(); throw new HttpRetryException("Cannot retry streamed HTTP body", response.code()); } if (! sameConnection(response, followUp.url())) { streamAllocation.release(); streamAllocation = new StreamAllocation(client.connectionPool(), createAddress(followUp.url()), call, eventListener, callStackTrace); this.streamAllocation = streamAllocation; } else if (streamAllocation.codec() ! = null) { throw new IllegalStateException("Closing the body of " + response + " didn't close its backing stream. Bad interceptor?" ); } request = followUp; priorResponse = response; }}Copy the code

In this case, the recovery mode is adopted according to the corresponding abnormal error through the method of exception capture, and the corresponding error state is recorded. However, after reaching the threshold, the pull-up retry operation is stopped. The network request failed.

BridgeInterceptor

A bridge between application code and network code that builds a network request based on a user’s request and, finally, a user response based on the network response. Some header parameters are set and processed accordingly. Such as GZIP compression issues and so on, construct the RequestHeader based on the pass parameters, RequestBody.

Public Response Intercept (Chain Chain) throws IOException {// Request body construction Request userRequest = chain.request(); Request.Builder requestBuilder = userRequest.newBuilder(); RequestBody body = userRequest.body(); if (body ! = null) { MediaType contentType = body.contentType(); if (contentType ! = null) { requestBuilder.header("Content-Type", contentType.toString()); } long contentLength = body.contentLength(); if (contentLength ! = -1) { requestBuilder.header("Content-Length", Long.toString(contentLength)); requestBuilder.removeHeader("Transfer-Encoding"); } else { requestBuilder.header("Transfer-Encoding", "chunked"); requestBuilder.removeHeader("Content-Length"); } } if (userRequest.header("Host") == null) { requestBuilder.header("Host", hostHeader(userRequest.url(), false)); } if (userRequest.header("Connection") == null) { requestBuilder.header("Connection", "Keep-Alive"); } //Gzip transform Boolean transparentGzip = false; if (userRequest.header("Accept-Encoding") == null && userRequest.header("Range") == null) { transparentGzip = true; requestBuilder.header("Accept-Encoding", "gzip"); } List<Cookie> cookies = cookieJar.loadForRequest(userRequest.url()); if (! cookies.isEmpty()) { requestBuilder.header("Cookie", cookieHeader(cookies)); } if (userRequest.header("User-Agent") == null) { requestBuilder.header("User-Agent", Version.userAgent()); Responsenetworkresponse = chain.proceed(requestBuilder.build()); HttpHeaders.receiveHeaders(cookieJar, userRequest.url(), networkResponse.headers()); Response.Builder responseBuilder = networkResponse.newBuilder() .request(userRequest); if (transparentGzip && "gzip".equalsIgnoreCase(networkResponse.header("Content-Encoding")) && HttpHeaders.hasBody(networkResponse)) { GzipSource responseBody = new GzipSource(networkResponse.body().source()); Headers strippedHeaders = networkResponse.headers().newBuilder() .removeAll("Content-Encoding") .removeAll("Content-Length") .build(); responseBuilder.headers(strippedHeaders); String contentType = networkResponse.header("Content-Type"); responseBuilder.body(new RealResponseBody(contentType, -1L, Okio.buffer(responseBody))); } return responseBuilder.build(); }Copy the code

As a bridge between the application layer and the network layer, its main purpose is to wrap the network request when it arrives at the network layer and wrap the response result when the response result of the network request comes back and transfer it to the user layer.

CacheInterceptor

Used to detect whether there is data in the cache, there is no change detection return, otherwise after the network request storage.

public Response intercept(Chain chain) throws IOException { Response cacheCandidate = cache ! = null ? cache.get(chain.request()) : null; long now = System.currentTimeMillis(); CacheStrategy Strategy = new cacheStrategy.factory (now, chain-.request (), cacheCandidate).get(); Request networkRequest = strategy.networkRequest; Response cacheResponse = strategy.cacheResponse; if (cache ! = null) { cache.trackResponse(strategy); } if (cacheCandidate ! = null && cacheResponse == null) { closeQuietly(cacheCandidate.body()); } // If we can neither make network requests nor cache, If (networkRequest == NULL && cacheResponse == null) {return new Response.Builder().request(chain.request()) .protocol(Protocol.HTTP_1_1) .code(504) .message("Unsatisfiable Request (only-if-cached)") .body(Util.EMPTY_RESPONSE) .sentRequestAtMillis(-1L) .receivedResponseAtMillis(System.currentTimeMillis()) .build(); } // If we don't need network requests, If (networkRequest == NULL) {return Cacheresponse.newBuilder ().Cacheresponse (stripBody(cacheResponse)) .build(); } // Execute the responsibility chain and obtain the Response result according to the network request. try { networkResponse = chain.proceed(networkRequest); } finally { // If we're crashing on I/O or otherwise, don't leak the cache body. if (networkResponse == null && cacheCandidate ! = null) { closeQuietly(cacheCandidate.body()); If (cacheResponse!) if (cacheResponse! = null) { if (networkResponse.code() == HTTP_NOT_MODIFIED) { Response response = cacheResponse.newBuilder() .headers(combine(cacheResponse.headers(), networkResponse.headers())) .sentRequestAtMillis(networkResponse.sentRequestAtMillis()) .receivedResponseAtMillis(networkResponse.receivedResponseAtMillis()) .cacheResponse(stripBody(cacheResponse)) .networkResponse(stripBody(networkResponse)) .build(); networkResponse.body().close(); cache.trackConditionalCacheHit(); cache.update(cacheResponse, response); return response; } else { closeQuietly(cacheResponse.body()); }} / / construct a Response body Response Response. = networkResponse newBuilder () cacheResponse (stripBody (cacheResponse)) .networkResponse(stripBody(networkResponse)) .build(); if (cache ! = null) { if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, networkRequest)) { // Offer this request to the cache. CacheRequest cacheRequest = cache.put(response); return cacheWritingResponse(cacheRequest, response); } if (HttpMethod.invalidatesCache(networkRequest.method())) { try { cache.remove(networkRequest); } catch (IOException ignored) { // The cache cannot be written. } } } return response; }Copy the code

ConnectInterceptor

Based on the passed data, a healthy connection is found and established, and some information such as timeout is set to the responding connection. If the connection is not established with the target host, the connection will be established with the target host.

public Response intercept(Chain chain) throws IOException { RealInterceptorChain realChain = (RealInterceptorChain) chain; Request request = realChain.request(); Transmitter transmitter = realChain.transmitter(); // We need the network to satisfy this request. Possibly for validating a conditional GET. boolean doExtensiveHealthChecks = ! request.method().equals("GET"); Exchange exchange = transmitter.newExchange(chain, doExtensiveHealthChecks); return realChain.proceed(request, transmitter, exchange); }Copy the code

The core of this is the Transmitter class, which acts as an intermediary between the network and the application. When you create a RealCall, you create this object. Next, what does the newExchange method do?

Exchange newExchange(Interceptor.Chain chain, boolean doExtensiveHealthChecks) {
  synchronized (connectionPool) {
    if (noMoreExchanges) {
      throw new IllegalStateException("released");
    }
    if (exchange != null) {
      throw new IllegalStateException("cannot make a new request because the previous response "
          + "is still open: please call response.close()");
    }
  }

  ExchangeCodec codec = exchangeFinder.find(client, chain, doExtensiveHealthChecks);
  Exchange result = new Exchange(this, call, eventListener, exchangeFinder, codec);

  synchronized (connectionPool) {
    this.exchange = result;
    this.exchangeRequestDone = false;
    this.exchangeResponseDone = false;
    return result;
  }
}
Copy the code

Exchange is a system that can carry new requests and responses, first found through ExchangeFinder.

RealConnection resultConnection = findHealthyConnection(connectTimeout, readTimeout,
  writeTimeout, pingIntervalMillis, connectionRetryEnabled, doExtensiveHealthChecks);
return resultConnection.newCodec(client, chain);
Copy the code

The core implementation in ExchangeFinder calls findHealthyConnection.

private RealConnection findHealthyConnection(int connectTimeout, int readTimeout, int writeTimeout, int pingIntervalMillis, boolean connectionRetryEnabled, boolean doExtensiveHealthChecks) throws IOException { while (true) { RealConnection candidate = findConnection(connectTimeout, readTimeout, writeTimeout, pingIntervalMillis, connectionRetryEnabled); // If this is a brand new connection, we can skip the extensive health checks. synchronized (connectionPool) { if (candidate.successCount == 0 && ! candidate.isMultiplexed()) { return candidate; } } // Do a (potentially slow) check to confirm that the pooled connection is still good. If it // isn't, take it out of the pool and start again. if (! candidate.isHealthy(doExtensiveHealthChecks)) { candidate.noNewExchanges(); continue; } return candidate; }}Copy the code

The method calls findConnecton repeatedly in an endless loop to find a suitable connection. FindConnection is the core implementation of connection multiplexing and connection establishment.

if (address.url().host().equals(this.route().address().url().host())) {
  return true; // This connection is a perfect match.
}
Copy the code

If any match is successful, how to return the current connection? If no match is found, try to establish a new connection.

result.connect(connectTimeout, readTimeout, writeTimeout, pingIntervalMillis, connectionRetryEnabled, call, eventListener); connectionPool.routeDatabase.connected(result.route()); Socket socket = null; synchronized (connectionPool) { connectingConnection = null; // Last attempt at connection coalescing, which only occurs if we attempted multiple // concurrent connections to the same host. if (connectionPool.transmitterAcquirePooledConnection(address, transmitter, routes, true)) { // We lost the race! Close the connection we created and return the pooled connection. result.noNewExchanges = true; socket = result.socket(); result = transmitter.connection; nextRouteToTry = selectedRoute; } else { connectionPool.put(result); transmitter.acquireConnectionNoEvents(result); }}Copy the code

Then the established new Connection will be added to the Connection pool for reuse, Connection established, will call its connect method to establish the Connection, first determine whether a tunnel is needed, if not, directly establish Socket Connection. After establishing a Socket connection, establishProtocol is invoked. The established Socket only establishes the TCP connection, and then performs subsequent protocol processing. At the same time, the corresponding event callback is performed through eventListner for each step.

if (route.requiresTunnel()) {
  connectTunnel(connectTimeout, readTimeout, writeTimeout, call, eventListener);
  if (rawSocket == null) {
    // We were unable to connect the tunnel but properly closed down our resources.
    break;
  }
} else {
  connectSocket(connectTimeout, readTimeout, call, eventListener);
}
establishProtocol(connectionSpecSelector, pingIntervalMillis, call, eventListener);
eventListener.connectEnd(call, route.socketAddress(), route.proxy(), protocol);
break;
}
Copy the code

After the Socket connection is established, the subsequent data can be written to the Socket through the corresponding wrapper, and the Socket is wrapped through Okio.

CallServerInterceptor

This interception is the last one in the chain of interceptions, the writing of the data, that is, the process of making a network request and interacting with the Server, and then returning the request data, and then going back to the call stack layer by layer, rewinding the data, and then doing the layer by layer processing.

@Override public Response intercept(Chain chain) throws IOException { RealInterceptorChain realChain = (RealInterceptorChain) chain; Exchange exchange = realChain.exchange(); Request request = realChain.request(); long sentRequestMillis = System.currentTimeMillis(); / / write head exchange request body. WriteRequestHeaders (request); / / write request body BufferedSink bufferedRequestBody = Okio. The buffer (exchange. CreateRequestBody (request, false)); request.body().writeTo(bufferedRequestBody); bufferedRequestBody.close(); / / read from the request body data response = response. NewBuilder (). The body (exchange. OpenResponseBody (response)). The build (); return response; }Copy the code

In this part, our core needs to pay attention to how our request data is written to the Socket, and how the data returned by the server is read by us. In connection part, we already know that after establishing a connection, there will be an InputStream and an OutputStream through Okio’s packaging, namely Okio’s Source and Sink. The above is a reduction code, omitting the implementation of some related logic, which mainly represents the writing of the request header, the writing of the request body, and then getting data from the returned data. The operation is essentially a wrapper around a write – in stream and a read – out stream of the Socket wrapper.

Related classes

  • Transmitter

In the middle of the OkHttp application layer and network layer, it can be used to cancel streams, you can cancel streams without affecting other streams in the shared connection pool. ConnectionPool, Call, and RealConnection are maintained.

  • ConnectionPool

To reduce latency, reuse of connections with the same address is implemented through a connectionPool. Multiple RealConnections are maintained internally through ArrayDeque.

  • Dispatcher

Each scheduler has an internal thread pool to schedule tasks for execution.

  • RealCall

Actual request execution

  • ExchangeCodec

Wrappers for Http requests and wrappers for Http request return results. It contains a Socket connection for http1 data transfer, also contains requests, and encapsulates read and write operations on the Socket.

  • RealConnection

When establishing a Socket connection, the Socket will be wrapped with OKIO, with a Sink, Source,

summary

This article is not depth profiling of the source code, probably still stay on the surface of the code of logic calls, for the use of design patterns, all kinds of technology, and its advantage compared with other network library, here temporarily for analysis, because the time to close this week, so for the performance, advantages, will in the next a depth analysis. The next article will also serve as a prelude to OkIO’s analysis.