Prologue of the preface

Hello, I’m Flywith24

It has been nearly two months since I posted on the Nuggets. Those of you who are familiar with me may know that I often separate my articles into different categories and write them according to series.

This is because I have always believed in two learning concepts:

  • Input forces output
  • Establish a systematic knowledge system

Also based on these two ideas, IN 2018, when I just graduated from college, I used one month’s salary to buy the HenCoder Plus series courses of Throwing line. At the end of the 19th, I started writing after I signed up for KunMinX’s Re-learning of Android.

On August 24th of this year, I launched a new series, Android Detail, dedicated to helping you build a systematic knowledge system. Just yesterday, the column had its 100th subscriber.

Now that the chatter is over, let’s begin this article.

preface

Nice to meet you! 👋

This article is the second in a series on processes that introduced some of the core concepts of Android processes, but continues along two lines.

The first part describes how memory is allocated in Android and how to manage memory when it is insufficient. The second part introduces the basis for cleaning up memory when it is low – process priority.

Knowing these things, then looking at the application life cycle, Activity life cycle and so on will have a different understanding.

As you read this article, you will know:

  • How does Android allocate memory between processes
  • How do I calculate the application memory usage
  • What are the two methods used by the system to free memory when it runs out of memory
  • Common process types
  • Priority of the process
  • ADJ with procstate
  • You can view the changes of ADJ and ProcState through operations and logs
  • I just want to know how the process is not dripping (Fan Wei teacher face 😆)
  • Analyze the disgusting behavior of rogue software through a case

Recommended reading

The following is a better read with this article. 😉

  • Graphical operating system memory management

    👆 highly recommended reading

  • Read the Android process priority adj. algorithm

Talk about Android’s design philosophy for memory usage

In this article we discussed the design philosophy of multitasking on Android. In order to maintain the best user experience, Android is designed to perform multiple tasks at once, in other words to allow multiple apps to run at the same time and switch between them quickly.

From a memory perspective, to achieve a “silky” switch, you must ensure that the corresponding process has been created and loaded into memory at the time of the switch. Ideally, we want all applications to be running. But in software engineering, “time” and “space” are always contradictory. If you want to get shorter “time” (silky experience), you have to pay more “space” (more memory). On the other hand, keeping an app running for long periods of time can consume more power, resulting in less battery life, which in turn affects the user experience.

Android memory management was designed against this paradoxical background. Instead of immediately killing used processes, the system caches previously created processes. If the memory on the device is insufficient, the memory can be reclaimed based on a certain policy. When the device memory reaches a certain threshold, the system kills processes according to the policy to release memory.

The first half of this article introduces the main structure of Android memory management, the management strategy when the memory is insufficient; The second part introduces the system according to which strategy to kill the process, and what methods to kill the process.

Memory allocation between processes

Memory type

Android devices contain three different types of memory: RAM, zRAM, and memory.

  • RAM is the fastest type of memory, but its size is usually limited. High-end devices typically have the largest RAM capacity
  • ZRAM is the RAM partition used for swap space. All data is compressed when it is put into zRAM and then decompressed when it is copied out of zRAM. This portion of RAM grows or shrinks as pages enter and exit zRAM. Device manufacturers can set a zRAM size upper limit
  • Storage contains all persistent data (such as file systems, etc.), as well as code added for all applications, libraries, and platforms. Memory is much larger than the other two types of memory. On Android, memory is not used for swap space as it is on other Linux implementations, because frequent writes can corrupt this memory and shorten the life of the storage medium

Memory pages

Android’s physical memory is divided into “pages.” Typically, each page has 4KB of memory.

Different types of pages serve different purposes:

  • Used Pages

    Memory pages that are actively used by the process

  • Cached Pages

    The memory page that is being used by the process. cached pages are backed up in storage and can be recycled if necessary

  • Free Pages

    Unused memory

Cache pages are divided into private pages and shared pages, which are respectively divided into clean pages and dirty pages:

  • Private page: Owned by a process and not shared

    • Clean page: An unmodified copy of a file in storage

    • Dirty pages: Modified copies of files in storage

  • Shared pages: used by multiple processes
    • Clean page: An unmodified copy of a file in storage
    • Dirty pages: Modified copies of files in storage

🌟 Note: Clean pages contain an exact copy of a file (or part of a file) that exists in storage. If a clean page no longer contains an exact backup of the file (for example, due to an application operation), it becomes a dirty page. Clean pages can be deleted because they can always be recreated using data from storage; Dirty pages cannot be deleted; otherwise, data will be lost.

Collecting Memory Usage

How do you know how much memory your application occupies?

As mentioned earlier, the device has paging management. The Linux kernel keeps track of which pages are being used by each process running on the device.

To count the memory footprint of an application, we simply count the number of pages the application is using. This process is a little more complicated because there are shared pages to consider. Applications that use the same service or library will share pages of memory. For example, the Google Play service and a game application might share location information services. This makes it difficult to determine how much memory belongs to the entire service and to each application.

Memory usage can be expressed in the following ways:

  • Resident Set Size – RSS

    Number of shared pages + non-shared pages used by the application

  • Proportional Set Size – PSS (Proportional Set Size – PSS)

    Number of unshared pages used by the application + number of evenly allocated shared pages (for example, if three processes share 3MB, the PSS of each process is 1MB)

  • Unique Set size-uss

    Number of non-shared pages used by the application (excluding shared pages)

The one we use most often is PSS.

We can use adb shell Dumpsys meminfo -s [process] to see the PROCESS’s PSS. Where process can be pid or applicationId.

🌟 Note: During memory optimization, it is not possible to simply compare the PSS value to determine whether the memory usage is optimized, because the PSS value is different for different devices, different configurations, different functions of the same application, and even for the same application in the same usage scenario due to different memory pressures.

In the figure above, the X-axis represents memory pressure, increasing from left to right, and the Y-axis represents PSS value.

The blue line represents the original APP, and the cyan represents the optimized APP.

As you can see, when the memory pressure is low, the PSS is relatively stable. As the memory pressure increases, KSWAPD starts to work and reclaim some cached pages, which may include the pages of the app process, so the PSS decreases. LMK is triggered when memory stress is extreme, and PSS becomes 0 (management when out of memory is covered in the next section).

We find two point samples, A and B, and the PSS of A is less than that of B, so we will get a better original app than the optimized one. This conclusion is clearly wrong.

A relatively accurate conclusion can be obtained by comparing PSS values under the same device memory pressure. Since memory pressure is difficult to control, the official recommendation is to test on devices with plenty of RAM so that memory pressure is kept at a low level and PSS values are stable with little fluctuation. To be able to determine more accurately whether the optimization is “negative optimization.”

Insufficient Memory Management

Linux has this Memory management strategy: OOM Killer (Out Of Memory Killer). This strategy is used to kill the process with the highest OOM_score when the split screen is out of memory.

Android has two main mechanisms for handling out-of-memory situations: kernel swap daemons and low-memory killers.

kernel swap daemon

The Kernel swap daemon (KSWAPD) is part of the Linux kernel that converts used memory to available memory. When the available memory on the device is low, the daemon becomes active. The Linux kernel maintains upper and lower thresholds for free memory. When the available memory falls below the lower threshold, kSWAPD starts to reclaim the memory. When the available memory reaches the upper threshold, KSWAPD stops reclaiming memory.

Kswapd can delete clean pages to reclaim memory because they are backed up and unmodified in storage. If a process tries to process a clean page that has been deleted, the system copies the page from storage to RAM. This operation is called request paging.

Kswapd can move cached private and anonymous dirty pages to zRAM for compression. This frees up free memory (available pages) in RAM. If a process tries to process a dirty page in zRAM, the page is decompressed and moved back to RAM. If the process associated with the compressed page is terminated, the page is removed from zRAM.

If the amount of available memory falls below a certain threshold, the system starts killing processes to reclaim the memory occupied by the process.

Low-memory killer

Many times, KSWAPD does not free enough memory for the system. In this case, the system uses onTrimMemory() to inform the application that it is out of memory and should reduce its allocation. If that’s not enough, the kernel starts killing processes to free up memory. It uses the low memory termination daemon (LMK) to perform this operation.

Unlike OOM Killer, LMK checks at regular intervals and starts working when the trigger threshold is reached.

So how does LMK kill processes? This leads to the concept of process type/process priority.

Common process types

To determine which processes should be killed when out of memory, Android puts each process into an “importance hierarchy” based on the components running in it and the state of those components. These process types include (in order of importance) :

Foreground Process

The foreground process is the process that the user needs to perform the current operation. The process is considered a foreground process if any of the following conditions are true:

  • The process runs an Activity that the user is interacting with. That is, the Activity’s onResume is called
  • The process is running a BroadcastReceiver, where the BroadcastReceiver onReceive method is executing
  • The process has a Service executing the code in onCreate, onStart, and onDestroy

These processes are the most important and have a limited number in the system. So the system tries to keep these processes running as well as possible. Unless memory is so low that even such processes cannot continue to run.

Visible Process

It can be seen that the process is running a task that the user knows about first, so terminating the process has a significant negative impact on the user experience. A process is considered visible if any of the following conditions are true:

  • The process is running an Activity that is visible to the user but not in the foreground (onPause is called)

    For example, application A’s process is A foreground process, but its foreground Activity is A dialog box followed by application B’s Activity. In this case, the process of application B is visible.

  • The process is running a foreground service started with startForground

  • The system is using its hosted services to implement user-aware specific functions, such as dynamic wallpaper, input method services, and so on

Such processes are considered so important that they are not terminated unless memory is too low to keep all foreground processes running properly.

Service Process

The Service process contains a Service that has been started using the startService method. Although these processes cannot be seen directly by the user, they are usually performing tasks that the user cares about (such as background network data uploads or downloads), so the system will always keep them running unless there is not enough memory to hold all foreground and visible processes.

Services that run for a long time (such as 30 minutes or more) may be demoted to the caching process described below. This prevents extremely long running services from taking up large amounts of memory due to memory leaks or other problems.

Cached process

The cache process is currently not needed, so the system is free to kill it if memory is needed elsewhere. To switch applications more efficiently, the system always keeps multiple cache processes available and periodically kills the earliest processes as needed. It is only in an emergency that the system reaches the point where it kills all cache processes and starts killing server processes.

In fact, the system divides process priorities in more detail, using oOM_score_adj.

Process priority

ADJ

In the LMK mechanism of Android, all processes are classified and each type of process has its oOM_adj value range. The higher the OOM_adj value is, the less important the process is. The higher the OOM_ADJ value is, the higher the oOM_ADJ value is, the lower the oOM_ADJ value is, the lower the oOM_ADJ value is. Process levels are defined as variables in ProcessList.java.

As of Android 7.0, ad_100, ad_200, ad_300. Previous versions of ADJ used the numbers 1, 2, and 3. Such adjustments can further refine the priorities of the process.

The following figure is based on the Android 11 source code.

The common process patterns described in the previous section are colored in the figure above.

PERCEPTIBLE_LOW_APP_ADJ is new for Android 10;

PERCEPTIBLE_RECENT_FOREGROUND_APP_ADJ is new in Android 9.

procstate

ADJ describes the process priority in terms of LMK, which is relatively low-level. In the Java world, the Activity Manager Service (AMS) manages the four major Components and processes of Android. AMS description of Process priority for procstate (the Process State), in the form of variables defined in the frameworks/base/core/Java/android/app/ActivityManager. In Java.

The following figure is based on the Android 11 source code.

🌟 varies slightly from version to version.

For example, PROCESS_STATE_FOREGROUND_SERVICE_LOCATION = 3 in Android 10. Android 11 removes this property and the values are successively advanced.

How do I query the process priority

adb shell dumpsys meminfo

Adb shell dumpsys meminfo adb shell dumpsys meminfo

adb shell dumpsys activity o

You can also use adb shell Dumpsys Activity O to query OOM information.

adb shell dumpsys activity p

You can also use adb shell Dumpsys Activity P to view detailed information about each process

Something a little more intuitive? 😍

The following figure uses an Android 10 device, so the procState value is slightly different from the above table, but the attribute names are the same.

According to log is because I will be in ActivityTaskManagerDebugConfig DEBUG_ALL opened, and don’t have friend of display device can be downloaded in the end of this article.

  1. Open the test app on the desktop with adj. = 0. At this time, the APP process is the foreground process and the process status in AMS is TOP

  2. Click home, and there are two instantaneous changes

    • Activity pause: adj. 200 indicates a user-aware process

    • Activity stop: adj = 700, belongs to the LAST application process (priority is higher than that of the cache process), and is in LAST state in AMS

  3. Open the gallery (any other application can be), at this time the test app ADJ has not changed, it is still the last application process. (Users can cut back to the test app from the recent task list or gesture operation)

  4. Click home again. The previous application process is the one where the gallery is located. Test app process adj. = 900, belonging to the cache process, AMS process state is CAC (cache process, including activity)

Now that you have an intuitive idea of what this is all about, you can try out more scenarios for yourself. 😉

I just want to know how the process is not 😝

Linux process killing mode

As we all know, interprocess communication has a method called “signaling”. Linux provides dozens of signals, each with different meanings. Signals are distinguished by their values. A signal can be sent to a process at any time, and the process needs to configure a signal handler for the signal. When a signal occurs, this function is executed by default. This is like a system emergency manual, when encounter what situation, do what things, are prepared in advance, out of the matter to do it.

Once a signal is generated, the user process does the following:

  • Perform default operations

    Linux has a default operation for each signal.

  • Capture the signal

    We can define a signal processing function for the signal. When the signal occurs, we execute the corresponding signal processing function.

  • Ignore the signal

    There are two signals that application processes cannot catch and ignore, namely SIGKILL and SEGSTOP, which are used to interrupt or terminate a process at any time.

The way Linux kills processes is through the SIGKILL signal, which has a value of 9.

Android bottom-layer process killing mode

Android is also the way to use signals at the bottom of the process.

Frameworks/base/core/Java/android/OS/Process. The Java USES three kinds of signal:

  • SIGNAL_QUIT = 3
  • SIGNAL_KILL = 9
  • SIGNAL_USR1 = 10

The class also encapsulates three process-killing static methods that eventually call the corresponding native methods. The bottom layer enters the kernel through system calls.

🌟 where killProcessQuiet and killProcessGroup are marked as @hide, app layer developers can only call killProcess(int PID). The only difference between killProcess and killProcessQuiet is that the former prints the log and the latter does not.

Upper-layer (AMS) processes are all calls to these three methods.

KillProcess is a static method that developers can call, but the app layer can only call it to commit suicide. If you can kill other processes at will, then “all hell breaks loose”.

Process.killProcess(Process.myPid())
Copy the code

As we mentioned in the previous section, signals with a signal value of 9 can neither be ignored nor captured, and are therefore handled directly by the kernel without further action. This is a bit like “The emperor let the minister die, I must die”, and did not hesitate a bit, did not take a cloud.

For signals 3 and 10, the SignalCatcher thread of the target process (ART VIRTUAL machine) is handed over to capture and complete corresponding operations.

For details of this part of the source code, see Gityuan’s understanding of the implementation principle of the kill process.

Upper AMS kill process mode

In the Java world, AMS encapsulates methods to kill processes, but these are essentially calls to the three kill methods of Process above.

SYSTEM_UID indicates that system sharedUserId is configured, and other permissions are related to permissions declared in the Manifest

The most powerful and effective method in the table above is forceStopPackage.

force stop

Force Stop is a powerful tool for killing processes in Android. It can be used to kill processes with specified package names, clear the four related components, and clear registered alarms and notifications.

Take adb shell am force-stop as an example to review the workflow of force Stop.

Adb shell am command invokes the frameworks/base/services/core/Java/com/android/server/am/ActivityManagerShellCommand onCommand Java Methods.

This method goes to a different branch depending on the CMD string passed in, and force-stop executes the runForceStop method, which ends up internally calling AMS’s forceStopPackage method. Its main code is as follows:

Let’s take a quick look at AMS’s forceStopPackage method:

This method has two main operations:

  1. Cleanup process, four major components
  2. Remove the Alarm Notification

Check out Gityuan’s Android process killer, forceStop. Although this article is based on 6.0 source code analysis, but the author compared the Android 11 source code, its core logic has not changed much, if you want to understand this part of the source code, this article is still a great reference value.

Mobile Settings – Application details forcibly stop, that is, force stop

Just look at the code is more abstract, let’s feel the power of force Stop more intuitively!

Feel the power of Force Stop

Multi-process architecture

We often separate the UI from the Service and use different processes during development. The purpose of this is to increase the process priority. The Service process that contains the Activity is different from the Service process that does not contain the Activity. However, Force Stop will kill all related processes of the application. Therefore, do not assume that a process can be separated and escape force Stop.

shareUserId

ShareUserId We introduced in the previous article. There are two important things about shareUserId:

  • Only two applications with the same signature (and requesting the same sharedUserId) can get the same user ID
  • Applications with the same shareUserId do not necessarily run in the same process

In the Gityuan Android process Killer –forceStop article, it was mentioned that using the same shareUserId creates a strong “together for life and death” relationship:

I don’t see any of this happening on Android 10 devices, so let me know in the comments.

Case study – Super Cleanup King

See such a rogue software in teacher Deng’s group recently, even if the user manually clicks forcibly stop, the software can restart.

This case has been explained by the MIUI guru at the end of this section. And this section we mainly introduce according to the previous content to analyze the software rogue behavior.

We use adb shell ps and filter the application UID to get the following results:

As you can see from the figure, the application has six processes:

The PPID (parent process ID) of the first four processes is 782 (the parent process is Zygote instead of Zygote64, which means it is a 32-bit application).

The last two processes (main, Daemons) have PPID 1 (init processes) and are native processes.

We then use adb shell am force-stop com.dn.cpyr. QLDS or manually kill the application process by clicking the “force-stop” button, and then print the process information again using ps.

We find that the application process is still in place, but the PID is different, which means that the previous process was killed, but then restarted.

We can verify this conclusion by checking the system logs.

As can be seen from the above log, the original process was indeed killed, and then the native process created by it sent crash, and then the new process was created to complete the rebirth.

I found this explanation in the group:

The picture above is from Teacher Deng’s wechat group

In fact, in order to achieve “unkillable”, the application is mainly processed in two directions:

  • Killed and rebooted, as mentioned above
  • Prioritize the process through various means

The author decompiled the application and sorted out the means to improve the process priority as follows:

  • UI process is separated from Service process

  • Use MediaPlayer to play silent music

  • Use AccountManager to back up data

  • Registered Accessibility Services (Ancillary Functions)

  • Register device manager

This kind of rogue software should directly send it an operation, never see again! 👎

conclusion

  • Android’s memory is divided into “pages,” each of which is about 4 KB

  • The most common way to look at the memory footprint of an application is to look at the PSS, which can be viewed using adb shell Dumpsys meminfo -s [process]

  • When memory reaches the threshold range for KSWAPD to operate, KSWAPD reclaims memory by deleting clean pages and compressing dirty pages

  • When memory reaches the operating threshold of LMK, LMK reclaims memory by killing processes

  • Processes have important priorities according to their importance. Processes with lower priorities are killed by LMK

  • On the upper level of the system, AMS manages processes, and the corresponding process priority is described as ProcState

  • The underlying Angle, whose process priority is described as ADJ

  • Linux uses signals to kill processes, and Android uses the same method. The source code is in process.java

  • Force Stop is very powerful, so don’t use so-called “dark technology” to occupy device memory