Introduction to the

Charles is a common network packet interception tool on PC. In mobile development, we often need to intercept network packets for analysis in order to debug the network communication protocol with the server. In addition to debugging ports for mobile development, Charles can also be used to analyze communication protocols for third-party applications. In conjunction with Charles’ SSL capabilities, Charles can also analyze the HTTPS protocol.

Charles implements interception and analysis of network packets by setting himself up as the network access proxy server of the system, through which all network access requests are processed.

The main functions of Charles include:

  • Intercepting HTTP and HTTPS network packets;
  • Support resending network requests to facilitate back-end debugging;
  • Support to modify network request parameters;
  • Support network request interception and dynamic modification;
  • Support analog slow network;

Download and install

Operating systems supported by Charles include:

  • Windows 64 – bit (msi);
  • Windows 32 bit (msi);
  • MacOS (DMG);
  • Linux 64 – bit (tar. Gz);
  • Linux 32 bit (tar. Gz);

Open the browser to visit the official website of Charles, download the Charles installation package of the corresponding system, and then install it.

Main Interface

The main interface of Charles is shown below:

Navigation bar

The top of Charles is the menu navigation bar, and the bottom is the tool navigation bar, as shown below:

The navigation bar provides the following common tools:

  • : Clears all captured requests.
  • : The red dot indicates that the request is being captured. The gray state indicates that no request is being captured.
  • : Grey indicates that network speed throttling is disabled, and green indicates that network speed throttling is enabled.
  • : Gray indicates that the breakpoint is not enabled. Red indicates that the breakpoint is enabled.
  • : Edit and modify the request, click to modify the content of the request;
  • : To send the request repeatedly, click the selected request will be sent again;
  • : verifies the response to the selected request;
  • : Common functions, including common functions in the Tools menu.
  • : Common Settings, including common Settings in the Proxy menu;

Main View

Charles provides two main views for viewing packets, called Structure and Sequence.

  • Structure: This view classifies network requests by domain names accessed;
  • Sequence: This view sorts network requests by access time;

You can switch back and forth between the two views depending on your needs. Charles provides a simple Filter function that allows you to enter a keyword to quickly Filter out web requests with a specified keyword in the URL.

For a specific network request, you can view the detailed request content and response content. If the request content is a POST form, Charles will automatically display the form for you. If the response is in JSON format, Charles can automatically format the JSON content for you to view. If the response content is an image, Charles can display a preview of the image.

Menu Bar Introduction

Charles’ main menus include: File, Edit, View, Proxy, Tools, Window, and Help. The most commonly used main menus are Proxy and Tools respectively, as shown in the figure below:

Proxy

Charles is an HTTP and SOCKS proxy server. Proxy requests and responses enable Charles to examine and change requests as they pass from the client to the server, and responses as they pass from the server to the client. Here are some of the proxy features provided by Charles. The view of the Proxy menu is shown below:

Recording Settings

Recording Settings are used together with Start/Stop Recording. You can configure the Recording behavior of a Charles session using the Start Recording Settings. The Recording Settings view is shown below:

Recording Settings Include Options, Include, and Exclude.

  • Options: Use Recording Size Limits to limit the Size of recorded data. When Charles records, the request, response header, and response body are stored in memory or written to temporary files on disk. Sometimes, the amount of data in memory may become too much, and Charles will notify you and stop recording. In this case, you should clear the Charles session to free up memory, and then start recording again. In recording Settings, you can limit the maximum size Charles will record; This won’t affect your browsing at all, Charles will just stop recording;
  • Include: Only requests that match the configured address are recorded.
  • Exclude: Only the requests matching the configured address will not be recorded.

The Include and Exclude tabs work the same way. Select Add and fill in the Procotol, Host, and Port information that you want to monitor to filter.

Throttle Settings

You can Throttle Settings with Start/Stop Throttling to configure Charles network speed simulation. The Throttle Settings view is shown below:

Select Enable Throttling to Enable network speed simulation and Throttle Preset to the network type. You can set Preset based on the actual situation. If you want to simulate a slow network for a specific site, check the Only for Selected Hosts TAB shown above and add the specified hosts TAB in the lower part of the dialog box.

The options in the Throttle Settings view have the following meanings:

  • Bandwidth: Bandwidth
  • Utilistation: percentage of utilization
  • Round-trip: indicates the round-trip delay
  • MTU: byte
  • Reliability: Reliability
  • Stability: stable
  • Unstable quality range: Unstable quality range
Breakpoint Settings

Breakpoint Settings and Enable/Disable Breakpoints. In Enable Breakpoints, Charles Breakpoint mode can be configured using Breakpoint Settings. The Breakpoint Settings view looks like this:

Check Enable Breakpoints to Enable breakpoint mode, select Add, and then fill in the Scheme, Procotol, Host, and Port information that you want to monitor. You can then observe or modify the content of the request or return, but you need to be aware of request timeouts in the process. Alternatively, you can set Breakpoints by right-clicking on a request url where you want to set Breakpoints.

Reverse Proxies Settings

A reverse proxy creates a Web server on a local port that transparently proxies requests to a remote Web server. All requests and responses on the reverse proxy can be logged in Charles. As shown below:

Reverse proxies can be useful if your client application does not support the use of HTTP proxies, or if you want to avoid configuring it to use proxies. Create a reverse proxy for the original target Web server, and then connect the client application to the local port; The reverse proxy is transparent to the client application, allowing you to see traffic that Charles might not have had access to before.

Port Forwarding Settings

Any TCP/IP or UDP Port can be configured to forward from Charles to a remote host using the Port Forwarding tool. This enables debugging of any protocol in Charles.

This is especially useful when debugging XMLSocket connections in Macromedia Flash.

You can also use Charles as a SOCKS proxy, so port forwarding is not required.

Windows Proxy(Logs all requests on your computer)

If you want to fetch requests from your computer, check the Windows Proxy option.

Deselect this option if you only need to grab the phone request.

Proxy Settings

The Proxy Settings view is shown below:

The default proxy port is 8888(which can be changed), and the Enable transparent HTTP proxying check box is used to Enable proxying on Charles.

Tools

Charles is an HTTP and SOCKS proxy server, and all requests go through Charles. Here are some of the utilities provided by Charles. The view of the Tools menu is shown below:

No Caching Settings

The No Caching tool prevents client applications, such as Web browsers, from Caching any resources. Therefore, always make a request to a remote site, and you can always see the latest version.

Application: This tool can be used for each request (Enable No Caching) or Only for the requests that you have configured (select Only for selected Locations when you Enable No Caching). When applied to selected requests, simple but powerful pattern matching can be used to limit the effect of the tool to the specified host and path.

How it works: The No Caching tool prevents Caching by manipulating HTTP request headers that control the response cache. Remove if-modified-since and if-none-match headers from the request and add Pragma: no-cache and cache-control: no-cache. Remove Expires, last-Modified, and ETag headers from the response and add Expires: 0 and cache-Control: no-cache.

Block Cookies Settings

The Block Cookies tool prevents Cookies from being sent and received. It can be used to test a website, just as cookies are disabled in a browser. Note that web crawlers (such as Google) generally do not support cookies, so the tool can also be used to simulate views of web crawler sites.

Scope of use: The tool can be used for each request (select Enable Block Cookies) or Only for the requests you configure (select Only for Selected Locations while enabling Block Cookies). When applied to selected requests, simple but powerful pattern matching can be used to limit the effect of the tool to the specified host and path.

How it works: The Block Cookies tool disables Cookies by manipulating HTTP request headers that control the response to Cookies. Removes the Cookie request header from the request to prevent Cookie values from being sent from the client application, such as a Web browser, to a remote server. Removes the set-cookie request header from the response to prevent the request from setting the Cookie received by the client application from the remote server.

Map Remote Settings

The Map Remote tool changes the request site based on the configured mapping to serve the response transparently from the new site as if it were the original request.

With this mapping, you can provide all or part of a site from another site. Such as:

  • Can put thexk72.com/charles/Map tolocalhost/charlesdev/forxk72.comProvide a subdirectory;
  • Can put thexk72.com/*.phpThis specifies the suffix for all files mapped tolocalhost/charlesdev/;

Usage advice: Map Remote is useful if you have a development version of your site and want to be able to browse the live site through some of the requests provided by your development. For example, you might want to provide the CSS and images directories from the development server. Map to dev.com/css/ or live.com/*.css to dev.com using live.com/css/ and so on.

Mapping type:

  • You can map directories to directories as followsxk72.com/charles/Map tolocalhost/charlesdev/;
  • You can map files to files as followsxk72.com/charles/download.phpMap toabc.com/testing/test.html;
  • You can map a directory with a file pattern to a directory, as inxk72.com/charles/*.php 到 localhost/charlesdev/;
  • If you do not specify a path in the target map, the path part of the URL will not change. If you want to map to the root directory, specify the target path field/At the end.

HTTPS: The Map Remote tool maps HTTP requests to HTTPS targets and vice versa, so you can Map HTTP or HTTPS sites to their opposite.

Site matching: Each site match may contain protocol, host, port, and path patterns to match a particular URL. Sites may include wildcards. As you add new sites to the tool, you may find more help on creating site matches.

Map Local Settings

The Map Local tool lets you use Local files as if they were part of a remote web site. You can develop files locally and test them as if they were online. The contents of the local file are returned to the client as if it were a normal remote response.

Map Local can greatly speed up development and testing that you would otherwise have to upload files to a web site to test results. With Map Local, you can test securely in your development environment.

Dynamic files: Dynamic files (such as files containing server-side scripts) are not executed by Map Local, so if there are any scripts in the file, the scripts will be returned to the browser as is, which may not be the expected result. If you want to use dynamic files as if they were part of a Remote web site, see the Map Remote tool.

How it works: When a request matches a Map Local Map, it checks for Local files that match the path. It does not include a query string (if any). If the requested file is found locally, it is returned as a response as if it had been loaded from a remote site, so it is transparent to the client. If the requested file is not found locally, the request is served by the web site as usual, returning data provided by the real server.

Site matching: Each site match may contain protocol, host, port, and path patterns to match a particular URL. Sites may include wildcards. As you add new sites to the tool, you may find more help on creating site matches.

Rewrite Settings

The Rewrite tool allows requests and responses to be created to modify their rules as they pass through Charles. For example, add or change header information, search and replace some text in the response content, and so on.

Rewrite sets: Rewrite sets can be activated and deactivated separately. Each collection contains a list of sites and rules. These sites select the requests and responses that the rules will run.

Rewrite rules: Each rule describes a rewrite operation. Rules may affect the Header, body, or part of the REQUESTED URL; It can act on requests or responses; It can define search, replace, or just replace style overrides.

Site matching: Each site match may contain protocol, host, port, and path patterns to match a particular URL. Sites may include wildcards. As you add new sites to the tool, you may find more help on creating site matches.

Debugging: Rewriting tools can be difficult to debug when rewriting operations do not work as expected. If you run into problems, try adding a very basic rule, such as one with obvious header information, so that you can see if the rule matches the request exactly. Also turn on debugging in the error log to get some debugging information printed from the error log accessed from the Window menu in Charles.

Black List Settings

The Black List tool allows you to enter domain names that should be blocked. When a Web browser tries to request any page from a blacklisted domain name, the request will be blocked by Charles. You can also enter wildcards to block subdomains.

Allow List Settings(Whitelist)

The Allow List tool allows you to enter only permitted domain names. The Allow List tool will block all requests except for whitelisted domain names.

The whitelist tool allows only specified domain names; Blacklist tool, used to block only specified domain names.

If a request matches both the Blacklist and whitelist, the request is blocked.

DNS Spoofing Settings(DNS Spoofing)

The DNS Spoofing tool allows you to trick DNS lookups by assigning your own host name to a remote address map. When a request goes through Charles, your DNS map takes precedence.

Charles contains a list of configured domain-to-IP address mappings. When a request is made for a listed domain name, the Spoof DNS plug-in finds that the spoofing IP redirects the request to that address. The host HTTP header stays the same, so just as your DNS server returns fraudulent IP.

A virtual host is one that has multiple sites on a single IP address, and the Web server determines the site to request based on the name typed in the browser. More precisely, it looks at the host header sent in the request.

If you don’t have DNS set up for your site, you generally can’t test it because you can’t just enter the IP address, because the server can’t get the name and therefore can’t connect the request to the site. Use DNS spoofing tools to overcome this problem.

Mirror Settings

When browsing a specific site, the Mirror tool copies the received response and saves it to the specified path on the disk.

The path to save the file will have the same directory structure as the site you browse, and Charles will create a root directory for the host name. The file name is exported from the URL and converted to the appropriate data for saving. The query string is contained in the file name. If you receive two responses from the same URL, the later file overwrites the previous file with the same name, so the response content stored in the image will always be the latest.

Selected site: The tool can be enabled for each request or only for specified sites. When used for selected sites, simple but powerful pattern matching can be used to limit the effect of the tool to a specified host and/or path.

Side effect: If mirroring is enabled for the request, it will cause any compressed or encoded response to be decoded. Therefore, if the server provides a compressed response, Charles will decompress it before passing it to the client, which usually has no impact. But if you have already built your own client, or if the client expects a compressed response, this will matter. Using a Web browser has no effect.

Auto Save Settings

The Auto Save tool automatically saves and clears recorded sessions at a set interval.

This is useful if you have Charles monitoring network activity for long periods of time and want to break down the records into manageable units, or avoid out-of-memory situations that can occur due to large amounts of data.

Enter the save interval in minutes and the directory to save the session file. You can choose whether to start the Auto Save tool every time you run Charles, otherwise the Auto Save tool will always be disabled when Charles starts.

Time stamps are stored in the name of the session file in the yyyyMMddHHmm format so that they appear in the correct order when sorted alphabetically.

Repeat (Repeat)

The Repeat tool allows you to select a request and Repeat it. Charles resends the request to the server and displays the response as a new request. This is useful if you are making back-end changes and want to test them with repeated requests in your browser (or other client). Especially if it takes some effort to recreate the request, such as getting points in the game, this can save a lot of effort.

Repeated requests are done internally in Charles, so the response cannot be viewed in a browser or other client; the response can only be viewed in Charles.

Repeat Advanced

The Repeat Advanced tool extends the Repeat tool with options for the number of iterations and concurrency. This is useful for load testing.

Validate (validation)

The Validate tool allows Charles to Validate recorded responses by sending them to the W3C HTML validator, W3C CSS validator, and W3C Feed validator.

The validation report is displayed in Charles and contains any warnings or errors associated with the corresponding line in the response source (double-click the line number in the error message to switch to the source view).

Because Charles tests the responses it records, it can test scenarios that are not easily tested, such as rendering an error message after submitting a form.

Revalidation: After validation, you can select the response from the validation results and Repeat, Repeat the original request, and then revalidate the results.

Publish Gist

The Publish Gist tool can Publish selected requests and responses as bullet points. By default, this bullet point will be posted anonymously, which means you won’t be able to delete it. You can authorize Charles to Publish using your GitHub account in the Publish Gist Settings of the Tools menu.

Import/Export Settings

The Import/Export tool allows you to Import/Export Charles’ Proxy, Tools, Preferences, and so on.

Profiles (configuration)

Profiles contains a full copy of all configuration Settings. Each time you change the current Settings, the system updates the current active profile, and when you change the active profile, all Settings are restored to the state they were when you last used the profile. Note that if you import a saved configuration, the Settings of the current configuration file will be overwritten. It is recommended to use import/export to back up or create snapshots of the current configuration and configuration files to maintain multiple parallel workspaces.

Using the tutorial

PC caught

Charles automatically configures the proxy Settings for the browser and tool, so opening the tool will automatically capture packets. Just make sure there are a few things:

  • Make sure Charles is in Start Recording state.
  • Check theProxy | Windows Proxy 和 Proxy | Mozilla FireFox Proxy;

Packet capture on the mobile terminal

The principle of mobile phone packet capture is similar to that of PC. The mobile phone entrusts the network to Charles for proxy to communicate with the server. The specific steps are as follows:

  • Make the mobile phone and computer in the same LAN, not necessarily the same IP segment, as long as they are under the same router;
  • PC configuration:
    • Turn off the firewall on your computer (this is important);
    • Open Charles agent function: through the main singles open Proxy | Proxy Settings window, fill in the Proxy port (default port is 8888, don’t need to modify), check the Enable transparent HTTP proxying.
    • If you don’t need to grab the request on the computer, you can uncheck the Proxy | Windows Proxy and the Proxy | Mozilla FireFox Proxy.
  • Mobile terminal configuration:
    • By Charles main menu Help | Local IP Address, or from the command line tools enter ipconfig check the machine’s IP Address;
    • Set proxy: Turn on the WIFI proxy Settings on the phone, enter the PC IP and Charles proxy port.
  • After setting up, we open any program that requires network request on the phone, and we can see Charles pop up the confirmation menu for requesting connection on the phone (only for the first time). Click Allow to complete the setting.
  • After completing the above steps, you can capture packets.

HTTPS packet capture on Windows

You need to configure HTTPS packet capture based on HTTP packet capture. The following steps need to be completed:

  • The HTTP packet capture configuration is complete.
  • PC installation, Charles Certificate: by Charles main menu Help | SSL Proxying | Install Charles Root Certificate installed Certificate;
  • Set the SSL Proxy: the main singles open Proxy | SSL Proxy Settings window, check the Enable SSL proxying.
  • Install the Charles certificate on the mobile terminal: By Charles main menu Help | SSL Proxying | Install Charles Root Certificate on a Mobile Device or Remote Browser installed Certificate;
  • After setting up, we open any program that requires network request on the phone, and we can see Charles pop up the confirmation menu for requesting connection on the phone (only for the first time). Click Allow to complete the setting.
  • After completing the preceding steps, you can capture packets using HTTPS.

HTTPS packet capture on MacOS

  • Install the certificate first.

  • Do not care about the pop-up prompt after installation;

  • A quick search for Charles certificates;

  • Set the trust certificate.

  • Restart the Charles;

Licenses

Effective test:

  • Registered Name: zhile.io
  • License Key: 48891cf209c6d32bf4

The resources

  • Charles feature introduction and use tutorial
  • Lesson 1: Charles
  • Charles tutorial (2)
  • Charles tutorial (3)
  • Charles tutorial (4)