One, foreword

When it comes to optimization, the first step is to divide a large function into small parts and then pick them out to find the points that can be optimized. The same is true for App network optimization.

When App accesses the network, DNS resolution is the first step of the network request. By default, we use the carrier’s LocalDNS service. According to statistics, it takes 200~300ms in this 3G network, and 100ms in 4G network.

Slow resolution is not the biggest problem with LocalDNS. There are some more serious problems, such as DNS hijacking, inaccurate DNS scheduling (caching, forwarding, NAT) resulting in performance degradation, etc. These are the most important problems to be solved by network optimization.

To optimize DNS, the simplest and most sophisticated solution now is to use HTTPDNS.

Today we’ll talk about DNS, HTTPDNS, and how to integrate HTTPDNS using OKHttp under Android.

DNS and HTTPDNS

2.1 What is DNS

Before we get to HTTPDNS, let’s take a quick look at what DNS is.

In the network world, every valid domain name has a server to provide services behind it, and the primary condition of our network communication is to know the IP address of the server.

But remembering domain names (web addresses) is certainly easier than remembering IP addresses. It would be very convenient if there was some way to look up the IP address of the server it was serving by domain name. This is where you need a DNS server and DNS resolution.

The Domain Name System (DNS) is used to locate IP addresses based on Domain names. It is the prerequisite for HTTP. The HTTP process can continue only after the domain name is correctly resolved into an IP address.

The DNS server must be highly available, concurrent, and distributed. It is divided into multiple hierarchies.

  • Root DNS server: Returns the IP address of the top-level DNS server.
  • Top-level DOMAIN DNS server: Returns the IP address of the authoritative DNS server.
  • Authoritative DNS server: Returns the IP address of the corresponding host.

The three types of DNS servers, like a tree structure, exist hierarchically.

When DNS resolution starts, if the LocalDNS server does not have cache, it sends a request to the LocalDNS server (usually the carrier). If the DNS server still does not have cache, it searches the corresponding top-level domain name from the root domain name, then searches the authoritative DNS server from the top-level domain name, and finally passes the authoritative DNS server. Obtain the IP address corresponding to the domain name.

In the process of providing domain name and IP address mapping, DNS actually provides many domain-based functions, such as server load balancing, but it also brings some problems.

2.2 DNS Problems

There are many details of DNS, this article will not expand on the details, its problems summed up is a few points.

1. The instability

DNS hijacking or failure causes service unavailability.

2. Not accurate

LocalDNS scheduling is not always based on the nearest principle. Some small carriers do not have a DNS server, so they directly call the DNS server of other carriers and transfer data across networks. For example, when a mobile carrier dispatches the IP address of the carrier on the user side, the access is slow or even restricted.

3. Not in time

The carrier may change the DNS TTL(time-to-live DNS cache Time), causing the DNS modification To take effect later.

In order to ensure the access quality of users on the network and reduce cross-network settlement, operators will set up content cache servers on the network. By forcibly pointing the domain name to the address of the content cache server, they can realize the purpose of keeping local traffic completely on the local network.

It’s not even consistent across carriers, and it’s a black box for us.

It is because of DNS problems, so pull out HTTPDNS.

2.3 HTTPDNS solution

DNS supports TCP as well as UDP, but most standard DNS uses UDP to interact with port 53 of the DNS server.

HTTPDNS, as its name implies, uses the HTTP protocol to interact with DNS servers. This avoids traditional DNS resolution and bypasses carrier’s LocalDNS server, effectively preventing domain name hijacking and improving domain name resolution efficiency.

This means that each family implements its own set of domain name resolution based on THE HTTP protocol and maintains an address book of domain names and IP addresses instead of using the same address book (DNS server).

It is said that wechat has its own deployment of NETDNS, and the major cloud service providers, Ali cloud and Tencent cloud also provide their own HTTPDNS service, for our ordinary developers, only need to pay a small fee, embedded in the mobile client SDK support HTTPDNS, can be used.

OKHttp access to HTTPDNS

Now that you understand the importance of HTTPDNS, let’s look at how to integrate HTTPDNS with OkHttp.

OkHttp is an open source project that handles web requests and is the hottest lightweight web framework on Android. In OkHttp, the default is to use the system’s DNS service, InetAddress, for domain name resolution.

InetAddress ip2= InetAddress.getByName("www.cxmydev.com");

System.out.println(ip2.getHostAddress());

System.out.println(ip2.getHostName());

Copy the code

There are two ways to use HTTPDNS in OkHttp.

  1. Using an interceptor, the domain name is replaced with an IP address before the request is sent.
  2. Provided through OkHttp.dns()Interface to configure HTTPDNS.

For both of these methods, the standard API is of course recommended. Interceptor way, also suggested to understand, the implementation is very simple, but there are pits.

3.1 Interceptor access Mode

1. Interceptor access

Interceptors are a very powerful mechanism in OkHttp that allows us to do some of our custom operations between requests and responses.

In OkHttp, you can customize an Interceptor by implementing the Interceptor interface. To use it, simply call the addInterceptor() method in the okHttpClient.Builder to register the interceptor.

The OkHttp interceptor is not the focus of this article, so let’s go back to interceptors to implement HTTPDNS.

class HTTPDNSInterceptor : Interceptor{

    override fun intercept(chain: Interceptor.Chain): Response {

        val originRequest = chain.request()

        val httpUrl = originRequest.url()



        val url = httpUrl.toString()

        val host = httpUrl.host()



        val hostIP = HttpDNS.getIpByHost(host)

        val builder = originRequest.newBuilder()



        if(hostIP! =null) {

            builder.url(HttpDNS.getIpUrl(url,host,hostIP))

            builder.header("host",hostIP)

        }

        val newRequest = builder.build()

        val newResponse = chain.proceed(newRequest)

        return newResponse

    }

}

Copy the code

In the interceptor, use the HttpDNS helper class to convert the Host to the corresponding IP through getIpByHost().

If caught by caught tools, you will find that was similar to the request of http://www.cxmydev.com/api/user, is replaced with: http://220.181.57.xxx/api/user.

2. Disadvantages of interceptor access

Using an interceptor directly bypasses the DNS step and replaces Host with the corresponding IP address before the request is sent.

This kind of scheme, in the process is very clear, there is no technical problem. However, there are some problems in this scheme, such as the certificate problem of IP direct connection under HTTPS, proxy problem, Cookie problem and so on.

One of the most serious problems is that when this scheme (interceptor +HTTPDNS) encounters HTTPS, if there is a server that supports multiple domain names, the certificate may not match.

Before we get to that, it’s important to understand HTTPS and SNI.

HTTPS ensures security. Before sending an HTTPS request, an SSL/TLS handshake is required. The handshake process is as follows:

  1. The client initiates a handshake request with parameters such as a random number and a list of supported algorithms.
  2. The server selects an appropriate algorithm to deliver a public key certificate and a random number.
  3. The client verifies the server certificate and sends random number information, which is encrypted using the public key.
  4. The server obtains random number information through the private key.
  5. Based on the interaction information, the two parties generate a Session Ticket, which is used as the encryption key for subsequent data transmission.

In this process, the client needs to verify the certificate issued by the server. First, unchain the certificate chain through the locally saved root certificate to ensure that the certificate is trusted. Then, the client also needs to check the domain domain and extension domain of the certificate to see whether the HOST of the request is included.

The problem with this step is that when using an interceptor, HOST is replaced with the IP resolved by HTTPDNS in the requested URL. When the server has multiple domain names and certificates, the server cannot determine which certificate should be returned when establishing SSL/TLS handshake. In this case, the policy may return the default certificate or not, which may cause a mismatch between the client and the certificate authentication domain. The SSL/TLS handshake fails.

This led to the SNI solution, which addresses SSL/TLS extensions that use multiple domain names and certificates for a single Server.

SNI works by sending the hostname of the site to be visited before connecting to the server to establish an SSL connection. The server returns the correct certificate based on this domain name. Today, most operating systems and browsers already support SNI extensions well.

3. Interceptor + HTTPDNS solution

This problem, in fact, there are solutions, here is a brief introduction.

To solve the problem of “domain mismatch “, you can perform the second step of hook certificate verification to replace the IP address with the original domain name and then perform certificate verification.

HttpURLConnect provides a HostnameVerifier interface, which can be implemented to complete the replacement.

public interface HostnameVerifier {

    public boolean verify(String hostname, SSLSession session);

}

Copy the code

If you use OkHttp, can reference OkHostnameVerifier (source: / / SRC/main/Java/okhttp3 / internal/TLS/OkHostnameVerifier. Java) implementation, to replace.

OkHttp itself does not recommend using interceptors to do HTTPDNS support, so here is not a discussion, here is only proposed to solve the idea, interested in studying the source code.

3.2 OKHttp standard API access

OkHttp already exposes a Dns interface. The default implementation uses the system’s InetAddress class to send UDP requests for Dns resolution.

We just need to implement the Dns interface of OkHttp to get support for HTTPDNS.

In our Dns interface implementation class, the Dns resolution method, replaced by HTTPDNS, will return the result of the resolution.

class HttpDns : Dns {

    override fun lookup(hostname: String): List<InetAddress> {

        val ip = HttpDnsHelper.getIpByHost(hostname)

        if(! TextUtils.isEmpty(ip)) {

            // Returns the list of addresses that it parses

            return InetAddress.getAllByName(ip).toList() 

        } else {

            // Parsing failed, using system parsing

            return Dns.SYSTEM.lookup(hostname)

        }

    }

}

Copy the code

It is also very simple to use, using the DNS () method at okhttp.build ().

mOkHttpClient = httpBuilder

        .dns(HttpDns())

        .build();

Copy the code

The benefits of this are:

  1. The domain name is still used for access, but the underlying DNS resolution has been changed to HTTPDNS to ensure that the IP address resolved is as expected.
  2. The HTTPS issue is resolved and the certificate is still validated using the domain name.

Since OkHttp exposes the DNS interface, we use it as much as possible.

Four, summary moment

Now you know, when doing App network optimization, the first step is to use HTTPDNS optimization DNS step.

All optimization is of course aimed at the end result, here two big factory public data, for Tencent’s products, after the access to HTTPDNS, the average user latency decreased by more than 10%, access failure rate decreased by more than a fifth. The Android hijacking rate of Baidu App’s Feed business decreased from 0.25% to 0.05%.

This optimization scheme, very dependent on HTTPDNS server, so it is recommended to use Ali Cloud, Tencent cloud such a relatively stable cloud service provider.

references:

[1] Baidu App network depth optimization series – DNS optimization

【 2 】 SIN:https://blog.csdn.net/makenothing/article/details/53292335

[3] Goose factory network matter, HTTPDNS service details

Public number background reply growth “growth”, will get the learning materials I prepared, can also reply “add group”, learning progress together; You can also reply to “questions” and ask me questions.

Recommended reading:

Do you need to know about a character encoding is here | illustration: HTTP request scope | Java exception handling | android to prevent the user closes the animation in animation failure | | Git retrieving lost code ali’s Alpha boost App start-up speed optimization