background

Packet capture is the most convenient and fastest way to determine whether the pot is in the front end. However, because many front-end applications can be run in the browser, it is most convenient to directly capture packets through Chrome. However, many students will not use the configuration proxy to capture packets. With the development of the front-end in the client, more and more front-end code running on the non-browser, such as THE micro channel small program I recently encountered is no way to run through the browser to capture packets, proxy capture has become a required option. This article records the experience of configuration agent after falling down many times, so as to learn from the problem more scientifically and quickly.

Write down this title, let me think of the primary school Chinese composition topic, unforgettable… Is it difficult to configure a Charles? The people who wrote the code would never admit it. But if the configuration can not catch the package, what is the problem, it will be a face meng force. Google, each answer with their own specious, try all the way over? Yes, I tried all the way in the past, and finally the problem was not solved, and I didn’t know what the problem was. The only thing I knew was that I had tried the answer, but it didn’t work, so I felt that it must be my keyword that wasn’t solved. Because the test students there can catch the package, their small program here with development tools, many times the configuration encountered difficulties give up halfway.

However, in the recent joint investigation of a project cooperated with a third party, the backend asked frantically, is it true that you did not send the request? I don’t know. It’s third-party code. I can only silently open the wechat development tool, check which branch is the code, sometimes my own account is not eligible, to find a suitable account, give him the authorization of wechat developers. After 10 minutes of a standard operation, I opened the page that needed to be checked. I told the back end that it was not the front end problem. However, I also have no confidence at this time, after all, some configuration of wechat development tools are not necessarily the same as online, and the code is not necessarily this version. It took a long time to get a satisfactory result. After a few rough interruptions to my happy coding, I decided to fight Charles to the death, and then persuaded the backend students to grab their own packages.

Principle of mobile phone packet capture

Client or server

Then I smoothly searched for an article on the Internet and completed the PROCESS of HTTPS packet capture according to the steps above. I opened wechat uneasily and saw the connection window of Charles pop up. I finally got over the difficulty.

Wechat or Android App

According to the basic principle of caught, cell phone package to forwarding agent on the port, so I doubt that WeChat is for the sake of safety, no agent for walking system, in order to test this idea, I open the browser and simple search on the mobile phone, enter the page crazy play “certificate of credibility”, according to the HTTPS encryption principle, this is the general, and in WeChat, None of the small programs can request an interface, wechat articles can be viewed normally without a certificate problem, so it is determined to be wechat, and it is the problem of wechat small programs.

Now that we’ve narrowed it down to a very small list, Google it and there are basically 10 or so different articles, all pointing to the same reason:

On Android7.0 and above, each application can define its own set of trusted cas. By default, an application trusts only the CA certificate preinstalled in the system, not the CA certificate installed by the user.

In the process of packet capture, whether Fiddler or Charles wants to capture HTTPS, the corresponding certificate must be installed on the mobile phone. The certificate installed through Fiddler /Charles is exactly the CA certificate installed by the user, so it is regarded as an insecure certificate.

It also reveals the following phenomena:

  • Android 7.0 or lower, regardless of the micro trust version, will trust the certificate provided by the system
  • Android 7.0 or later, wechat 7.0 * or later, wechat will trust the certificate provided by the system
  • Android 7.0 or later, wechat 7.0 or later, wechat only trusts its own list of certificates

Install the old version of wechat, try and really can parse the data. At this point, the problem is clear

Terrible HTTPS

Packet capture is to see the content of the communication, and HTTPS confidentiality is required that others can not see the content, naturally is a conflict, fortunately, recently read some HTTPS security and man-in-the-middle attack articles, to proxy capture this middleman have a certain understanding, in short, HTTPS encryption process is:

  • When a client requests a certificate from the server, the server returns a certificate with a public key.
  • After verifying the validity of the server certificate, the client generates a random number and sends it to the server through the public key
  • Subsequently, the client and server use the exchanged random numbers to encrypt and decrypt data symmetrically

See an HTTPS article for more details, such as HTTP and HTTPS details. To capture HTTPS packets, the corresponding certificate must be installed on the mobile phone. This certificate is generated by fiddler or Charles, and the proxy exchanges keys with the client for encrypted connection. Therefore, the proxy can view the communication content of the client.

  • A certificate can be generated by anyone, but only the certificate generated by the authority can be trusted. The browser or client has some trusted authority certificates built-in before delivery. A secure certificate can match the signature of these authority certificates during certificate validity verification
  • The agent uses a certificate generated by a non-authoritative authority, validates that the validity cannot be matched to an authoritative authority, and considers the communication to be insecure (which it is), so an untrusted message pops up.

Two things must be done in order for data to be exchanged smoothly during HTTPS packet capture:

  • A certificate is required to extract the public key
  • The client application is informed that the certificate is not valid and has the opportunity to complete symmetric key exchange

After knowing this, others told me that XXX can catch HTTPS packets on Android, I can go straight to the topic to catch wechat small programs, no longer tangled whether this agent where there is a configuration of black technology I have not unlocked, and some so-called can be in Android 7.0 above the package capture scheme, Just focus on how it solves the certificate trust problem, rather than trying it out one by one.

The solution

It is easier to know where the problem is and how to avoid it. For the certificate problem, you can find ways to solve it in the certificate location and wechat version.

  • A class of parallel space software that can hijack application behavior and intercept certificate validity verification is theoretically optimal, but it cannot be opened by trying wechat at present.

  • Root put the agent certificate into the system certificate, after asking the big guy, now android 8.0 or above can not root, so there is no solution, who has a license plate can point out, but I heard that root will be detected to seal the number, or not very safe.

  • Wechat version rollback, many places on the network can download the old version of wechat, but the interface capacity of the small program is more dependent on the wechat version, the old version of wechat will have compatibility problems.

  • Just use apples

conclusion

Finally, in the face of Google’s massive data, trying to answer without understanding the principle is just exhaustive. Recalling this process, I began to have a vague understanding of HTTPS, and had no clue if anything went wrong. At the beginning, I still suspected that I could not install the certificate, and THAT I was not familiar with Charles and did not match it correctly.

Understanding the principle, can be very sure according to the principle step by step to narrow the scope of the problem, coupled with practical verification, in order to get a reliable answer, each question appears, we understand the details of it is limited, and the principle is universal. Think about the search keywords [MAC Charles phone packet capture] [Charles HTTPS certificate]… And now very confident search [Android wechat 7 small programs can not catch packets]. When I communicated with my colleague before, it was said that whistle can catch the bag on the mobile phone. I asked him to help me configure it. If I failed, I suspected there was something wrong with the configuration.

In front of the problem, I am only a pupil. But this question made me grow up a little bit: Google is a brute force, pruning brute force with principles, and algorithms don’t just apply to code, they apply to everything