\ \//
                          \\  .ooo.  //. @ @ @ @ @ @ @ @ @. : @ @ @ @ @ @ @ @ @ @ @ @ @ : : @ @.'@ @ @ @ @'. @ @ : @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ : @ @ : @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @. @ @ : @ @ @'@@@@@@@@@@@@@@@@@, @@@
                   @@@ '@ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @, @ @ @ @ @ @'@@@@@@@@@@@@@@@@@, @@@
                   @@@ '@ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @, @ @ @ @ @ @'@@@@@@@@@@@@@@@@@, @@@
                   @@@ '@ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @, @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @'@ @ @ @ @ @ @ @ @ @ @ @ @ @ @'
                           @@@@   @@@@
                           @@@@   @@@@
                           @@@@   @@@@
                           '@ @'   '@ @': @@@.. @@@@@@@: + @@' @@@@ '@@@@.@@@@'@ @ @ @ : + @ @ ` @ @ @ @ ` @ @ @ @ @ @ @ @ @ @ + @ @ ` @ @ @ @ ` @ @ @ @. @ @ @ @ : + @ @ @ @ @ ` @ @ @ @ ` @ @ @ @ @ @ @ @ @ @ @ @ @ @. @ @ @ @ @ @ @ @ @ @ @ + @ @ @ @ @ ` @ @ @ @ ` @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ + @ @ @ @ @ ` @ @ @ @ @ @ @ @ @ @ ` @ @ @ @ @ @ @ : @ @ @ @ @ @ @ @ + @ @ @ @ @ ` @ @ @ @ @ @ @ @ @ @ ` @ @ @ @ @ @ # @ @ + @ @ @ @ @ @ + + ` @ @ @ @ @ @ @ @ @ ` @ @ @ @ @ @ : @ @ # @ @ : . @ @ ` + @ @ @ + @ @ ` @ @ @ @ ` @ @ @ @ @ @ # @ @ + @ @ @.. @ @ @ + @ @ @ @ @ ` @ @ @ @ ` @ @ @ @ @ @ @, @ @ @ @ @ @ @ @ @ @ @ + @ @ @ @ @ ` @ @ @ @ ` @ @ @ @ @ @ @ @ @ @ @ @ # @ @ @ @ @@@@@@@ +@@ #@@ '@@@@' @@@@: @@@@: @@'@ @ @ @ @ @ @ : @ @ : @ @ :Copy the code

For Android Developer, many open source libraries are essential knowledge points for development, from the use of ways to implementation principles to source code parsing, which require us to have a certain degree of understanding and application ability. So I’m going to write a series of articles about source code analysis and practice of open source libraries, the initial target is EventBus, ARouter, LeakCanary, Retrofit, Glide, OkHttp, Coil and other seven well-known open source libraries, hope to help you 😇😇

Official account: byte array

Article Series Navigation:

  • Tripartite library source notes (1) -EventBus source detailed explanation
  • Tripartite library source notes (2) -EventBus itself to implement one
  • Three party library source notes (3) -ARouter source detailed explanation
  • Third party library source notes (4) -ARouter own implementation
  • Three database source notes (5) -LeakCanary source detailed explanation
  • Tripartite Library source note (6) -LeakCanary Read on
  • Tripartite library source notes (7) -Retrofit source detailed explanation
  • Tripartite library source notes (8) -Retrofit in combination with LiveData
  • Three party library source notes (9) -Glide source detailed explanation
  • Tripartite library source notes (10) -Glide you may not know the knowledge point
  • Three party library source notes (11) -OkHttp source details
  • Tripartite library source notes (12) -OkHttp/Retrofit development debugger
  • Third party library source notes (13) – may be the first network Coil source analysis article

This article is based on the latest version of OkHttp. It is worth mentioning that OkHttp and OkIO have been officially rewritten in Kotlin language, so the students who have not learned Kotlin may even be relatively difficult to understand the source code, Kotlin introduction can see my article: twenty-six thousand words with you Kotlin introduction

dependencies {
    implementation 'com. Squareup. Okhttp3: okhttp: 4.9.0'
}
Copy the code

Let’s start with a small example, and the rest of the presentation will be based on the modules involved in this example

/ * * * the author: leavesC * time: 2020/11/10 [* description: * GitHub:https://github.com/leavesC * /
const val URL = "https://publicobject.com/helloworld.txt"

fun main(a) {
    val okHttClient = OkHttpClient.Builder()
        .connectTimeout(Duration.ofSeconds(10))
        .readTimeout(Duration.ofSeconds(10))
        .writeTimeout(Duration.ofSeconds(10))
        .retryOnConnectionFailure(true)
        .build()
    val request = Request.Builder().url(URL).build()
    val call = okHttClient.newCall(request)
    valresponse = call.execute() println(response.body? .string()) }Copy the code

The above code completes a Get request with the following actions:

  1. OkHttpClient is obtained through Builder mode. OkHttpClient contains global configuration information for network requests, including link timeout, read/write timeout, link failure retry and other configurations
  2. The Request is obtained in Builder mode. The Request contains all the Request parameters of the network Request, including URL, method, HEADERS, and body
  3. The newCall method is used to get a Call. A Call is used to initiate a request, which can be used to execute, enqueue, cancel, and other operations
  4. Call the execute method to initiate the synchronization request and return a Response object. The Response object contains all the information returned from the network request. If the request fails, this method will throw an exception
  5. Get the body of the Response object and read it as a string stream. The printed result is the Android robot Egg at the beginning of the article

A, OkHttpClient

OkHttpClient uses Builder mode to complete the initialization. It provides many configuration parameters. Each option has a default value, but most of the time we need to customize it, so it is necessary to know all the parameters

class Builder constructor() {
    / / scheduler
    internal var dispatcher: Dispatcher = Dispatcher()
    / / the connection pool
    internal var connectionPool: ConnectionPool = ConnectionPool()
    // The list of interceptors
    internal val interceptors: MutableList<Interceptor> = mutableListOf()
    // List of network interceptors
    internal val networkInterceptors: MutableList<Interceptor> = mutableListOf()
    // Event listening
    internal var eventListenerFactory: EventListener.Factory = EventListener.NONE.asFactory()
    // Whether to retry when the connection fails
    internal var retryOnConnectionFailure = true 
    // Source server authentication
    internal var authenticator: Authenticator = Authenticator.NONE
    // Whether to allow redirection
    internal var followRedirects = true
    // Whether to allow SSL redirection
    internal var followSslRedirects = true
    //Cookie
    internal var cookieJar: CookieJar = CookieJar.NO_COOKIES
    / / cache
    internal var cache: Cache? = null
    //DNS
    internal var dns: Dns = Dns.SYSTEM
    / / agent
    internal var proxy: Proxy? = null
    // Proxy selector
    internal var proxySelector: ProxySelector? = null
    // Proxy authentication
    internal var proxyAuthenticator: Authenticator = Authenticator.NONE
    / / Socket factory
    internal var socketFactory: SocketFactory = SocketFactory.getDefault()
    // Secure socket layer
    internal var sslSocketFactoryOrNull: SSLSocketFactory? = null
    internal var x509TrustManagerOrNull: X509TrustManager? = null
    internal var connectionSpecs: List<ConnectionSpec> = DEFAULT_CONNECTION_SPECS
    / / HTTP protocol
    internal var protocols: List<Protocol> = DEFAULT_PROTOCOLS
    // Confirm the host name
    internal var hostnameVerifier: HostnameVerifier = OkHostnameVerifier
    / / certificate chain
    internal var certificatePinner: CertificatePinner = CertificatePinner.DEFAULT
    internal var certificateChainCleaner: CertificateChainCleaner? = null
    internal var callTimeout = 0
    internal var connectTimeout = 10 _000
    / / read timeout
    internal var readTimeout = 10 _000
    / / write timeout
    internal var writeTimeout = 10 _000
    // The interval between pings
    internal var pingInterval = 0
    internal var minWebSocketMessageToCompress = RealWebSocket.DEFAULT_MINIMUM_DEFLATE_SIZE
    internal var routeDatabase: RouteDatabase? = null
}
Copy the code

Second, the Request

Request contains all the parameters of a network Request, including the following five:

  1. The url. This is the address of the network request and the possible query key-value pairs
  2. Method. The request mode can be GET, HEAD, POST, DELETE, PUT, or PATCH
  3. Headers. The request header can be used to store tokens, timestamps, and so on
  4. The body. The request body
  5. Tags. Use to uniquely identify this request
    internal var url: HttpUrl? = null
    internal var method: String
    internal var headers: Headers.Builder
    internal var body: RequestBody? = null
    /** A mutable map of tags, or an immutable empty map if we don't have any. */
    internal var tags: MutableMap<Class<*>, Any> = mutableMapOf()
Copy the code

Third, the Call

When you Call okHttClient.newCall(request), you get a Call object

  /** Prepares the [request] to be executed at some point in the future. */
  override fun newCall(request: Request): Call = RealCall(this, request, forWebSocket = false)
Copy the code

Call is an interface that we can think of as the initiator of network requests. It can be used to make synchronous or asynchronous requests, but will throw an exception if repeated requests are made more than once

interface Call : Cloneable {
    
  // Return the Request object for this network Request
  fun request(a): Request
    
  // Initiates a synchronization request, which may throw an exception
  @Throws(IOException::class)
  fun execute(a): Response
    
  // Make an asynchronous request and call the final result back and forth through the Callback
  fun enqueue(responseCallback: Callback)

  // Cancel network request
  fun cancel(a)

  // Whether a request has been made
  fun isExecuted(a): Boolean

  // Whether the request has been canceled
  fun isCanceled(a): Boolean
  
  // Timeout calculation
  fun timeout(a): Timeout

  // The same Call is not allowed to be made repeatedly. To make a Call again, you can use this method to get a new Call object
  public override fun clone(a): Call

  fun interface Factory {
    fun newCall(request: Request): Call
  }
}
Copy the code

The actual type returned by the newCall method is RealCall, which is the only implementation class of the Call interface

When we call the execute method to initiate a synchronization request, the main logic is:

  1. Interpret whether the request is repeated
  2. The event log
  3. Add itself to the Dispatcher and remove itself from the Dispatcher at the end of the request
  4. throughgetResponseWithInterceptorChainMethod gets the Response object
class RealCall(
  val client: OkHttpClient,
  /** The application's original request unadulterated by redirects or auth headers. */
  val originalRequest: Request,
  val forWebSocket: Boolean
) : Call {
    
  override fun execute(a): Response {
    check(executed.compareAndSet(false.true)) { "Already Executed" }
    timeout.enter()
    callStart()
    try {
      client.dispatcher.executed(this)
      return getResponseWithInterceptorChain()
    } finally {
      client.dispatcher.finished(this)}}}Copy the code

Fourth, the Dispatcher

Can see from the above analysis, getResponseWithInterceptorChain method is the key to its return we finally get the Response. But instead of going into that method, let’s look at the logic of the Dispatcher

A Dispatcher is a scheduler used to cache and dispatch network requests globally. It contains the following member variables

  var maxRequests = 64

  var maxRequestsPerHost = 5

  /** Ready async calls in the order they'll be run. */
  private val readyAsyncCalls = ArrayDeque<AsyncCall>()

  /** Running asynchronous calls. Includes canceled calls that haven't finished yet. */
  private val runningAsyncCalls = ArrayDeque<AsyncCall>()

  /** Running synchronous calls. Includes canceled calls that haven't finished yet. */
  private val runningSyncCalls = ArrayDeque<RealCall>()
Copy the code
  • MaxRequests. Maximum number of threads allowed to concurrently execute network requests at one time
  • MaxRequestsPerHost. Maximum number of simultaneous requests under the same host
  • ReadyAsyncCalls. Saves the asynchronous task that is currently waiting to be executed
  • RunningAsyncCalls. Save the asynchronous task that is currently being executed
  • RunningSyncCalls. Save the current synchronization task

A client should not initiate an unlimited number of network requests at the same time, because in addition to network resources, system resources are limited. Each request needs to be executed by one thread, and the number of concurrent execution threads supported by the system is limited. So OkHttp internally uses maxRequests to control the maximum number of threads that can execute asynchronous requests simultaneously. In addition, to improve efficiency, OkHttp allows multiple network requests to the same host to share the same Socket, and the maximum number of shared is maxRequestsPerHost

To count the above two runtime parameters, you need to use readyAsyncCalls, runningAsyncCalls, and runningSyncCalls to save the network requests that are currently executing or about to be executed. RunningSyncCalls is used to save the current synchronization task that is being executed. It stores realcalls. ReadyAsyncCalls and runningAsyncCalls are used to store asynchronous tasks, which store AsyncCall

1. Synchronize requests

The execute() method of a RealCall sends itself to the Dispatcher before starting the request and is removed from the Dispatcher when the request is complete

class RealCall(
  val client: OkHttpClient,
  /** The application's original request unadulterated by redirects or auth headers. */
  val originalRequest: Request,
  val forWebSocket: Boolean
) : Call {
 
  override fun execute(a): Response {
    check(executed.compareAndSet(false.true)) { "Already Executed" }
    timeout.enter()
    callStart()
    try {
        // Add to dispatcher
      client.dispatcher.executed(this)
      return getResponseWithInterceptorChain()
    } finally {
        // Remove from dispatcher
      client.dispatcher.finished(this)}}}Copy the code

The Dispatcher internally only adds realcalls to or removes them from runningSyncCalls accordingly, The purpose of saving to runningSyncCalls is to make it easy to count the total number of all currently running requests and to be able to cancel all requests. In addition, because synchronous requests run directly on the caller’s thread, synchronous requests are not constrained by maxRequests

class Dispatcher constructor() {
    
      /** Used by [Call.execute] to signal it is in-flight. */
  	  @Synchronized 
      internal fun executed(call: RealCall) {
    	runningSyncCalls.add(call)
      }
    
      /** Used by [Call.execute] to signal completion. */
  	  internal fun finished(call: RealCall) {
    	finished(runningSyncCalls, call)
  	  }

  	  private fun <T> finished(calls: Deque<T>, call: T) {
    	val idleCallback: Runnable?
    	synchronized(this) {
      		if(! calls.remove(call))throw AssertionError("Call wasn't in-flight!")
      		idleCallback = this.idleCallback
    	}
        // Determine whether there is a network request that needs to be processed
    	val isRunning = promoteAndExecute()
    	if(! isRunning && idleCallback ! =null) {
      		idleCallback.run()
    	}
     }
    
}
Copy the code

2. Asynchronous request

The RealCall enqueue method wraps the incoming Callback as an AsyncCall object and passes it to the Dispatcher

class RealCall(
  val client: OkHttpClient,
  /** The application's original request unadulterated by redirects or auth headers. */
  val originalRequest: Request,
  val forWebSocket: Boolean
) : Call {
    
  override fun enqueue(responseCallback: Callback) {
    check(executed.compareAndSet(false.true)) { "Already Executed" }
    callStart()
    client.dispatcher.enqueue(AsyncCall(responseCallback))
  }
    
}
Copy the code

Since enqueue is an asynchronous request, OkHttp needs to construct its own thread to execute the request, and then Callback the result to the external through the Callback. The asynchronous request logic is carried by the AsyncCall class

AsyncCall is a non-static inner class of RealCall, so AsyncCall has access to all variables and methods of RealCall. In addition, AsyncCall inherits the Runnable interface and its executeOn method is used to pass in a thread pool object to execute the run method. In the run method or call the getResponseWithInterceptorChain () method to get the response, and will execute the results through the Callback (regardless of success or failure) to go back to bring up, It also removes itself from the Dispatcher after the request is completed

  internal inner class AsyncCall(
    private val responseCallback: Callback
  ) : Runnable {
    @Volatile var callsPerHost = AtomicInteger(0)
      private set

    fun reuseCallsPerHostFrom(other: AsyncCall) {
      this.callsPerHost = other.callsPerHost
    }

    fun executeOn(executorService: ExecutorService) {
      client.dispatcher.assertThreadDoesntHoldLock()
      var success = false
      try {
        executorService.execute(this)
        success = true
      } catch (e: RejectedExecutionException) {
        val ioException = InterruptedIOException("executor rejected")
        ioException.initCause(e)
        noMoreExchanges(ioException)
        responseCallback.onFailure(this@RealCall, ioException)
      } finally {
        if(! success) { client.dispatcher.finished(this) // This call is no longer running!}}}override fun run(a) {
      threadName("OkHttp ${redactedUrl()}") {
        var signalledCallback = false
        timeout.enter()
        try {
          val response = getResponseWithInterceptorChain()
          signalledCallback = true
          responseCallback.onResponse(this@RealCall, response)
        } catch (e: IOException) {
          if (signalledCallback) {
            // Do not signal the callback twice!
            Platform.get().log("Callback failure for ${toLoggableString()}", Platform.INFO, e)
          } else {
            responseCallback.onFailure(this@RealCall, e)
          }
        } catch (t: Throwable) {
          cancel()
          if(! signalledCallback) {val canceledException = IOException("canceled due to $t")
            canceledException.addSuppressed(t)
            responseCallback.onFailure(this@RealCall, canceledException)
          }
          throw t
        } finally {
          client.dispatcher.finished(this)}}}}Copy the code

The Dispatcher will save the AsyncCall object to readyAsyncCalls and then use findExistingCallWithHost to find out if there is currently an asynchronous request to the same Host. If so, we exchange the callsPerHost variable, which is used to mark the current number of requests to the same Host, and finally call the promoteAndExecute method to determine whether requests are currently allowed

class Dispatcher constructor() {
 
   internal fun enqueue(call: AsyncCall) {
    synchronized(this) {
      readyAsyncCalls.add(call)

      // Mutate the AsyncCall so that it shares the AtomicInteger of an existing running call to
      // the same host.
      if(! call.call.forWebSocket) {// Check whether there are currently asynchronous requests to the same Host
        val existingCall = findExistingCallWithHost(call.host)
        if(existingCall ! =null) call.reuseCallsPerHostFrom(existingCall)
      }
    }
    promoteAndExecute()
  }

  private fun findExistingCallWithHost(host: String): AsyncCall? {
    for (existingCall in runningAsyncCalls) {
      if (existingCall.host == host) return existingCall
    }
    for (existingCall in readyAsyncCalls) {
      if (existingCall.host == host) return existingCall
    }
    return null}}Copy the code

The promoteAndExecute() method is used to get all current run-eligible requests from the readyAsyncCalls list because the total number of network requests being executed may have reached a limit or the number of requests to the same Host may have reached a limit. The request is stored in runningAsyncCalls and the thread pool is called for execution

private fun promoteAndExecute(a): Boolean {
    this.assertThreadDoesntHoldLock()

    val executableCalls = mutableListOf<AsyncCall>()
    val isRunning: Boolean
    synchronized(this) {
      val i = readyAsyncCalls.iterator()
      while (i.hasNext()) {
        val asyncCall = i.next()
	    // If the total number of asynchronous requests currently being executed exceeds the limit, return directly
        if (runningAsyncCalls.size >= this.maxRequests) break // Max capacity.
        // If the total number of requests to the same Host exceeds the limit, the next request is taken
        if (asyncCall.callsPerHost.get() > =this.maxRequestsPerHost) continue // Host max capacity.

        i.remove()
        // Add callsPerHost by one, indicating that the number of links to the Host is increased by one
        asyncCall.callsPerHost.incrementAndGet()
        // Save asyncCall in the executable list
        executableCalls.add(asyncCall)
        // Save asyncCall to the executing list
        runningAsyncCalls.add(asyncCall)
      }
      isRunning = runningCallsCount() > 0
    }

    // Execute all qualified requests
    for (i in 0 until executableCalls.size) {
      val asyncCall = executableCalls[i]
      asyncCall.executeOn(executorService)
    }

    return isRunning
  }
Copy the code

3, ArrayDeque

As mentioned above, the storage container for the three requests is an ArrayDeque. ArrayDeque is a non-thread-safe dual-ended queue, so external active thread synchronization is required whenever multi-threaded operations are involved. So let’s think about why OkHttp chose ArrayDeque as the task container. In my shallow view, there are the following:

  • Internally, ArrayDeque uses an array structure to store data, with elements in clear order, in line with our basic expectation of first-come, first-executed network requests
  • When selecting asynchronous requests that meet the running conditions, we need to iterate over readyAsyncCalls, and the array is more efficient in traversal
  • After iterating to a qualifying request, the request needs to be removed from readyAsyncCalls and transferred to runningAsyncCalls, and ArrayDeque, as a dual-ended queue, is more memory efficient
  • The Dispatcher is in a multi-threaded environment and requires thread synchronization itself. Choosing ArrayDeque, a non-thread-safe container, can eliminate the unnecessary thread synchronization cost

Thread pool

OkHttp’s asynchronous requests are handled by its internal thread pool, which looks like this:

  private var executorServiceOrNull: ExecutorService? = null

  @get:Synchronized
  @get:JvmName("executorService") val executorService: ExecutorService
    get() {
      if (executorServiceOrNull == null) {
        executorServiceOrNull = ThreadPoolExecutor(0.Int.MAX_VALUE, 60, TimeUnit.SECONDS,
            SynchronousQueue(), threadFactory("$okHttpName Dispatcher".false))}return executorServiceOrNull!!
    }
Copy the code

What are the advantages of this thread pool parameter setting? In my shallow view, there are two points:

  1. The number of core threads is 0, and the thread timeout is 60 seconds. Note If a thread is idle for 60 seconds when there is no task to be executed, the thread will be reclaimed. This prevents idle threads from wasting system resources, which is suitable for situations where resources of mobile devices are scarce
  2. The maximum number of threads allowed is Int.MAX_VALUE, which can be considered completely unrestricted, and the task queue is SynchronousQueue. The feature of SynchronousQueue is that when a task is enqueued, it must wait for it to be consumed otherwise the enqueued operation will be blocked. Since the pool allows an unlimited number of threads, each enqueued task can be immediately transferred to a thread (either to an idle thread or to a new thread). This ensures timely processing of tasks, in line with our expectation that network requests should be initiated and completed as soon as possible

Although the thread pool itself has almost no limit on the maximum number of threads, since the operation of submitting tasks is controlled by maxRequests, the pool actually runs maxRequests threads at most at the same time

5. Drive request execution

Since the internal thread pool of OkHttp does not allow an unlimited number of new threads to execute requests, when the total number of requests reaches Max, subsequent requests must be in the wait state first. When will these requests be started?

Both synchronous and asynchronous requests are called to the Two Finished methods of the Dispatcher, from which the promoteAndExecute() method is triggered to traverse the task list for execution, thus promoting the execution of the task list. Therefore, requests in the Dispatcher can be viewed as being initiated spontaneously, with the completion of each request automatically triggering the execution of the next request (if any), eliminating the need for periodic checks

  /** Used by [AsyncCall.run] to signal completion. */
  internal fun finished(call: AsyncCall) {
    call.callsPerHost.decrementAndGet()
    finished(runningAsyncCalls, call)
  }

  /** Used by [Call.execute] to signal completion. */
  internal fun finished(call: RealCall) {
    finished(runningSyncCalls, call)
  }

  private fun <T> finished(calls: Deque<T>, call: T) {
    val idleCallback: Runnable?
    synchronized(this) {
      if(! calls.remove(call))throw AssertionError("Call wasn't in-flight!")
      idleCallback = this.idleCallback
    }
	// Check whether there is a task that can be started. If there is, start the task
    val isRunning = promoteAndExecute()

    if(! isRunning && idleCallback ! =null) {
      idleCallback.run()
    }
  }
Copy the code

6, summary

  • In the case of synchronous requests, the network request process is completed directly on the caller’s thread and is not controlled by the Dispatcher
  • If the request is asynchronous, the request is stored in readyAsyncCalls. Whether the request can be initiated immediately is subject to both maxRequests and maxRequestsPerHost. If so, it is fetched from readyAsyncCalls and stored in runningAsyncCalls, which is then handed over to the internal thread pool of OkHttp for execution
  • Whether the external request is synchronous or asynchronous, the internal request is by invocationgetResponseWithInterceptorChain()Method to get the Response
  • The thread pool inside the Dispatcher itself allows Int.MAX_VALUE threads to run simultaneously, but the actual number of threads is controlled by maxRequests

Fifth, RealInterceptorChain

Key again to see getResponseWithInterceptorChain () method, its main logic is through interceptors to complete the online request process. In this method, the following interceptors are added by default, in addition to the interceptors that are actively set externally

  1. RetryAndFollowUpInterceptor. Responsible for failed retries and redirects
  2. BridgeInterceptor. Responsible for converting the Request constructed by the user, adding the necessary headers and cookies, and performing GZIP decompression if necessary after receiving the response
  3. CacheInterceptor. For handling the cache
  4. ConnectInterceptor. Responsible for establishing a connection with the server
  5. CallServerInterceptor. Responsible for sending requests to and receiving data from the server

Finally, the request and interceptors are used to generate a RealInterceptorChain object that returns the response

  @Throws(IOException::class)
  internal fun getResponseWithInterceptorChain(a): Response {
    // Build a full stack of interceptors.
    val interceptors = mutableListOf<Interceptor>()
    // Add an interceptor set by the developer
    interceptors += client.interceptors
    
    // Add a default interceptor
    interceptors += RetryAndFollowUpInterceptor(client)
    interceptors += BridgeInterceptor(client.cookieJar)
    interceptors += CacheInterceptor(client.cache)
    interceptors += ConnectInterceptor
      
    if(! forWebSocket) {// If not WebSocket, add NetworkInterceptors set by the developer
      interceptors += client.networkInterceptors
    }
      
    //CallServerInterceptor is where the network request is actually made
    interceptors += CallServerInterceptor(forWebSocket)

    val chain = RealInterceptorChain(
        call = this,
        interceptors = interceptors,
        index = 0,
        exchange = null,
        request = originalRequest,
        connectTimeoutMillis = client.connectTimeoutMillis,
        readTimeoutMillis = client.readTimeoutMillis,
        writeTimeoutMillis = client.writeTimeoutMillis
    )

    var calledNoMoreExchanges = false
    try {
      val response = chain.proceed(originalRequest)
      if (isCanceled()) {
        response.closeQuietly()
        throw IOException("Canceled")}return response
    } catch (e: IOException) {
      calledNoMoreExchanges = true
      throw noMoreExchanges(e) as Throwable
    } finally {
      if(! calledNoMoreExchanges) { noMoreExchanges(null)}}}Copy the code

Interceptor is an important part of OkHttp, which provides developers with a high degree of freedom. The Interceptor interface itself contains only an intercept method, in which the original Request object and the final Response are obtained

fun interface Interceptor {
   @Throws(IOException::class)
   fun intercept(chain: Chain): Response   
}
Copy the code

For example, we can customize a LogInterceptor that prints the request parameters of a network request and the final return value

class LogInterceptor : Interceptor {

    override fun intercept(chain: Interceptor.Chain): Response {
        val request = chain.request()
        println(request)
        val response = chain.proceed(request)
        println(response)
        return response
    }

}
Copy the code

The purpose of Interceptor is to provide developers with a way to control the initiation and end of a network request, such as adding headers, logging, intercepting requests, and modifying the ResponseBody. Each Interceptor is only responsible for the operations it cares about. Therefore, there is bound to be a need to add multiple Interceptors

As we know, only after each Interceptor processes the request in turn, OkHttp can request a response from the network according to the final request object. Therefore, each Interceptor needs to get the request in turn for custom processing. After the response is requested, the Interceptor may also need to process the response, so the response needs to be passed to each Interceptor in turn. So, how to implement multiple Interceptor to the series?

Here is a simplified version of Interceptor implementation ideas

Let’s assume that our own Interceptor implementation class has two: LogInterceptor and HeaderInterceptor, here is simply to get the request and response time to print out, the focus is to see the order of each Interceptor calls. In order to concatenate the two interceptors, the RealInterceptorChain loops to obtain the next Interceptor that index points to. Every time to build a new RealInterceptorChain object as a parameter to invoke the intercept method, such external just call a RealInterceptorChain. Proceed can get the final response object

/** * Author: leavesC * Time: 2020/11/11 16:08 * Description: */
class Request

class Response

interface Chain {

    fun request(a): Request

    fun proceed(request: Request): Response

}

interface Interceptor {

    fun intercept(chain: Chain): Response

}

class RealInterceptorChain(
    private val request: Request,
    private val interceptors: List<Interceptor>,
    private val index: Int
) : Chain {

    private fun copy(index: Int): RealInterceptorChain {
        return RealInterceptorChain(request, interceptors, index)
    }

    override fun request(a): Request {
        return request
    }

    override fun proceed(request: Request): Response {
        val next = copy(index = index + 1)
        val interceptor = interceptors[index]
        val response = interceptor.intercept(next)
        return response
    }

}

class LogInterceptor : Interceptor {
    override fun intercept(chain: Chain): Response {
        val request = chain.request()
        println("LogInterceptor -- getRequest")
        val response = chain.proceed(request)
        println("LogInterceptor ---- getResponse")
        return response
    }
}

class HeaderInterceptor : Interceptor {
    override fun intercept(chain: Chain): Response {
        val request = chain.request()
        println("HeaderInterceptor -- getRequest")
        val response = chain.proceed(request)
        println("HeaderInterceptor ---- getResponse")
        return response
    }
}

fun main(a) {
    val interceptorList = mutableListOf<Interceptor>()
    interceptorList.add(LogInterceptor())
    interceptorList.add(HeaderInterceptor())
    val request = Request()
    val realInterceptorChain = RealInterceptorChain(request, interceptorList, 0)
    val response = realInterceptorChain.proceed(request)
    println("main response")}Copy the code

Look at the code above ideas can also, but when will find out IndexOutOfBoundsException after running, because the code is not crossing the line judge index. In addition, the above code also lacks a real place to generate the Response object. Each Interceptor is merely a relay call, so it needs a place to actually complete the network request and return the Response object. Namely CallServerInterceptor

Therefore, the Intercept method of the CallServerInterceptor is used to actually perform the network request and generate the Response object. You cannot call the proceed method here. The CallServerInterceptor needs to be placed at the end of the interceptorList

class CallServerInterceptor : Interceptor {
    override fun intercept(chain: Chain): Response {
        val request = chain.request()
        println("CallServerInterceptor -- getRequest")
        val response = Response()
        println("CallServerInterceptor ---- getResponse")
        return response
    }
}

fun main(a) {
    val interceptorList = mutableListOf<Interceptor>()
    interceptorList.add(LogInterceptor())
    interceptorList.add(HeaderInterceptor())
    interceptorList.add(CallServerInterceptor())
    val request = Request()
    val realInterceptorChain = RealInterceptorChain(request, interceptorList, 0)
    val response = realInterceptorChain.proceed(request)
    println("main response")}Copy the code

The final result is shown below. As you can see, the Intercept method is called in the order in which it was added, while response is passed in the reverse direction

LogInterceptor -- getRequest
HeaderInterceptor -- getRequest
CallServerInterceptor -- getRequest
CallServerInterceptor ---- getResponse
HeaderInterceptor ---- getResponse
LogInterceptor ---- getResponse
main response
Copy the code

The above code simplifies the way OkHttp implements RealInterceptorChain. In essence, multiple interceptors are called layer by layer in the way of chain of responsibility. After the last interceptor finishes processing, the result will be passed to the next interceptor. After the last interceptor (CallServerInterceptor) completes processing, the Response is passed one layer above the other

class RealInterceptorChain(
  internal val call: RealCall,
  private val interceptors: List<Interceptor>,
  private val index: Int.internal valexchange: Exchange? .internal val request: Request,
  internal val connectTimeoutMillis: Int.internal val readTimeoutMillis: Int.internal val writeTimeoutMillis: Int
) : Interceptor.Chain {
    
  internal fun copy(
    index: Int = this.index,
    exchange: Exchange? = this.exchange,
    request: Request = this.request,
    connectTimeoutMillis: Int = this.connectTimeoutMillis,
    readTimeoutMillis: Int = this.readTimeoutMillis,
    writeTimeoutMillis: Int = this.writeTimeoutMillis
  ) = RealInterceptorChain(call, interceptors, index, exchange, request, connectTimeoutMillis,
      readTimeoutMillis, writeTimeoutMillis)
    
  @Throws(IOException::class)
  override fun proceed(request: Request): the Response {...val next = copy(index = index + 1, request = request)
    val interceptor = interceptors[index]
    @Suppress("USELESS_ELVIS")
    val response = interceptor.intercept(next) ?: throw NullPointerException(
        "interceptor $interceptor returned null")...return response
  }
    
}
Copy the code

Six, Interceptor

When we build OkHttpClient, there are two types of methods to add interceptors: addInterceptor and addNetworkInterceptor

    val okHttClient = OkHttpClient.Builder()
        .addInterceptor { chain ->
            chain.proceed(chain.request())
        }
        .addNetworkInterceptor { chain ->
            chain.proceed(chain.request())
        }
        .build()
Copy the code

Interceptor and NetworkInterceptor are called application interceptors and network interceptors respectively, so what is the difference between them?

As mentioned earlier, OkHttp executes interceptors in the following order, which determines the timing difference between interceptors

	val interceptors = mutableListOf<Interceptor>()
    interceptors += client.interceptors
    interceptors += RetryAndFollowUpInterceptor(client)
    interceptors += BridgeInterceptor(client.cookieJar)
    interceptors += CacheInterceptor(client.cache)
    interceptors += ConnectInterceptor
    if(! forWebSocket) { interceptors += client.networkInterceptors } interceptors += CallServerInterceptor(forWebSocket)Copy the code
  • Because the application is in the list of interceptors in the head, so interceptors will be applied in the whole responsibility link first implemented, even after happened in RetryAndFollowUpInterceptor redirection request failure retry or network, etc., using interceptor also will be triggered only once, but the network interceptor is called many times
  • The network interceptor is located after the CacheInterceptor, so when the CacheInterceptor hits the cache, it will not execute the network request, and the network interceptor will not be called, so the network interceptor can be short-circuited. In addition, the network interceptor is located after the ConnectInterceptor, and the network link is prepared before the network interceptor is invoked, indicating that the network interceptor itself is associated with the actual network request logic
  • From the point of view of the single request flow, the application interceptor being called does not mean that a network request has been made, but the network interceptor being called means that a network request has been made. So if we want to use interceptors to log the details of network requests, we need to consider the difference in call timing: the application interceptors are not aware of some of the headers that OkHttp automatically adds, but the network interceptors are; Unless the application interceptor actively interrupts the request, each request must be executed, but the network interceptor may be short-circuited

Borrow an official picture to show

Here’s an example of how square officials can monitor the download progress of a 10 MB image with an interceptor and download the image to the system’s desktop

The idea is to add a layer of proxy to the original ResponseBody and calculate the percentage between the number of bytes read from the network and the contentLength of the resource to determine the download progress. In addition, since the interceptor is related to the exact network request, it makes sense to set it as a network interceptor

/ * * * the author: leavesC * time: 2020/11/14 15:49 * description: * GitHub:https://github.com/leavesC * /
fun main(a) {
    run()
}

interface ProgressListener {
    fun update(bytesRead: Long, contentLength: Long, done: Boolean)
}

private fun run(a) {
    val request = Request.Builder()
        .url("https://images.pexels.com/photos/5177790/pexels-photo-5177790.jpeg")
        .build()
    val progressListener: ProgressListener = object : ProgressListener {
        var firstUpdate = true
        override fun update(bytesRead: Long, contentLength: Long, done: Boolean) {
            if (done) {
                println("completed")}else {
                if (firstUpdate) {
                    firstUpdate = false
                    if (contentLength == -1L) {
                        println("content-length: unknown")}else {
                        System.out.format("content-length: %d\n", contentLength)
                    }
                }
                println(bytesRead)
                if(contentLength ! = -1L) {
                    System.out.format("%d%% done\n".100 * bytesRead / contentLength)
                }
            }
        }
    }
    val client = OkHttpClient.Builder()
        .addNetworkInterceptor { chain: Interceptor.Chain ->
            valoriginalResponse = chain.proceed(chain.request()) originalResponse.newBuilder() .body(ProgressResponseBody(originalResponse.body!! , progressListener)) .build() } .build() client.newCall(request).execute().use { response ->if(! response.isSuccessful) {throw IOException("Unexpected code $response")}val desktopDir = FileSystemView.getFileSystemView().homeDirectory
        val imageFile = File(desktopDir, "${System.currentTimeMillis()}.jpeg")
        imageFile.createNewFile()
        // Read the InputStream and write it to the image fileresponse.body!! .byteStream().copyTo(imageFile.outputStream()) } }private class ProgressResponseBody constructor(
    private val responseBody: ResponseBody,
    private val progressListener: ProgressListener
) : ResponseBody() {

    private var bufferedSource: BufferedSource? = null

    override fun contentType(a): MediaType? {
        return responseBody.contentType()
    }

    override fun contentLength(a): Long {
        return responseBody.contentLength()
    }

    override fun source(a): BufferedSource {
        if (bufferedSource == null) {
            bufferedSource = source(responseBody.source()).buffer()
        }
        return bufferedSource!!
    }

    private fun source(source: Source): Source {

        return object : ForwardingSource(source) {

            var totalBytesRead = 0L

            @Throws(IOException::class)
            override fun read(sink: Buffer, byteCount: Long): Long {
                val bytesRead = super.read(sink, byteCount)
                // read() returns the number of bytes read, or -1 if this source is exhausted.
                totalBytesRead += if(bytesRead ! = -1L) bytesRead else 0
                progressListener.update(totalBytesRead, responseBody.contentLength(), bytesRead == -1L)
                return bytesRead
            }
        }
    }

}
Copy the code

The progress output looks like this:

content-length: 11448857
467
0% done
1836
0% done
3205...99% done
11442570
99% done
11448857
100% done
completed
Copy the code

At the end,

This is the end of the OkHttp source code, but what is missing from this article is the ConnectInterceptor and CallServerInterceptor, where OkHttp completes the actual network request. I’m going to skip over the lower-level areas like Connections and sockets because I can’t explain them clearly enough

OkHttp is very efficient, but it is still primitive in use. In general, we still need a layer of encapsulation on top of OkHttp. Retrofit is an excellent encapsulation library for OkHttp. Tripartite library source notes (7) -Retrofit source detailed explanation

The next article will be about the OkHttp interceptor in action