Welcome to the Wechat official account: BaronTalk

A preface.

This article I brew for a long time, reference a lot of information, read a lot of source code, but still dare not write. For fear of their understanding on the deviation, we cause misunderstanding, laughing. I am afraid that MY understanding is not thorough enough to accurately express Binder’s design essence in clear and straightforward words. I still write with trepidation to this day.

Binder’s complexity is far more than can be explained in a single article. The purpose of this article is to look at Binder’s design from a higher dimension and ultimately help shape a complete concept. For the application layer development of students, to understand the degree of this article is about the same. For a deeper understanding of the Binder implementation mechanism, read the resources and related source code at the end of this article.

Binder overview

A brief introduction to what Binder is. Binder is an interprocess communication mechanism based on OpenBinder. OpenBinder was originally developed by Be Inc., and later by Plam Inc. Take over. Binder means glue. The name means to bond with different processes so they can communicate. A more comprehensive definition of Binder will be explained in detail after we have introduced the communication principles of Binder.

2.1 Why must Binder be understood?

As an Android engineer, do you often wonder:

  • Why do we need serialization to pass objects between activities?
  • What is the process for launching an Activity?
  • What is the underlying communication mechanism of the four components?
  • What is the internal implementation of AIDL?
  • Where should plug-in programming techniques start? And so on…

All of these questions are relevant to Binder, and it is necessary to understand the Bidner communication mechanism.

We know that An Android application consists of one or more of four components: Activity, Service, Broadcast Receiver, and Content Provide. Sometimes these components run in the same process, sometimes in different processes. Communication between these processes relies on the Binder IPC mechanism. In addition, Android provides various services for application layer, such as ActivityManagerService and PackageManagerService, which are implemented based on Binder IPC mechanism. The Binder mechanism is an important place in Android, and it’s no exaggeration to say that understanding Binder is the first step toward advanced engineering on Android.

2.2 Why Binder?

The Android system is based on the Linux kernel, which already provides IPC mechanisms such as pipes, message queues, shared memory and sockets. So why does Android provide Binder to implement IPC? For performance, stability, and security reasons.

performance

First, the performance benefits. Socket is a universal interface with low transmission efficiency and high overhead. It is mainly used for inter-process communication across the network and low-speed communication between processes on the local machine. The message queue and pipeline adopt the store-and-forward mode, that is, data is copied from the sender cache to the kernel cache, and then from the kernel cache to the receiver cache, at least two copy processes. Although shared memory does not need to be copied, it is complex to control and difficult to use. Binder requires only one data copy and is second only to shared memory in performance.

Note: Data copy times of various IPC modes. This table comes from Android Binder design and Implementation – Design

The IPC way Number of data copies
The Shared memory 0
Binder 1
Socket/ pipe/message queue 2

The stability of

In terms of stability, Binder is based on C/S architecture, which allows the Client to fulfill any requirements to the Server. Binder’s architecture is clear, responsibilities are clear, and mutual independence makes it more stable. Shared memory does not need to be copied, but it is controlled and difficult to use. In terms of stability, the Binder mechanism is superior to memory sharing.

security

Another aspect is security. As an open platform, Android has a huge variety of applications on the market for users to choose and install. Therefore, security is extremely important for Android platform. As users, of course, we do not want our downloaded APP to secretly read my contacts, upload my private data, steal the traffic and consume the battery of the phone. Traditional IPC has no security measures and relies entirely on upper-level protocols for assurance. Firstly, the traditional IPC receiver cannot obtain the reliable process user ID/ process ID (UID/PID) of the other party, so it cannot authenticate the other party. Android assigns its own UID to each installed APP, so a process’s UID is an important indicator of its identity. Traditional IPC can only be filled by the user in the data packet UID/PID, but this is unreliable, easy to be exploited by malicious programs. Reliable identity is only added in the kernel by the IPC mechanism. Second, the traditional IPC access point is open, as long as you know these access points of the program and the peer end can establish a connection, no matter how to prevent malicious programs through guessing the receiver address to obtain a connection. At the same time, Binder supports both real-name Binder and anonymous Binder, which has high security.

For these reasons, Android needs to establish a new IPC mechanism to meet the stability, transport performance, and security requirements of the system. This is called Binder.

Finally, a table summarizes Binder’s advantages:

advantage describe
performance Only one data copy is required, second only to shared memory in performance
The stability of Based on THE C/S structure, the responsibilities are clear and the structure is clear, so the stability is good
security Assign a UID to each APP. A process’s UID is an important marker for identifying a process

Three. Linux under the traditional interprocess communication principle

Understanding concepts and principles related to Linux IPC helps us understand Binder communication principles. Therefore, before introducing Binder’s cross-process communication principles, let’s talk about how traditional interprocess communication is implemented on Linux systems.

3.1 Basic Concepts

We’ll start with some of the basic concepts involved in interprocess communication in Linux, and then work our way up to show you how traditional interprocess communication works.

The figure above shows some of the basic concepts involved in cross-process communication in Liunx:

  • Process isolation
  • Process Space partition: User Space/Kernel Space
  • System call: user mode/kernel mode

Process isolation

Simply put, memory is not shared between processes in the operating system. The two processes are like two parallel worlds, and process A has no direct access to process B’s data. This is A popular interpretation of process isolation. Data exchange between process A and process B requires A special communication mechanism: inter-process communication (IPC).

Process Space partition: User Space/Kernel Space

Today’s operating systems use virtual storage, and for 32-bit systems, its addressing space (virtual storage space) is 2 to the power of 32, which is 4GB. The core of the operating system is the kernel, which is independent of ordinary applications and has access to protected memory space as well as permissions to underlying hardware devices. To prevent User processes from directly operating the Kernel and ensure Kernel security, the operating system logically divides the virtual Space into User Space and Kernel Space. For The Linux operating system, the highest 1GB byte is used by the kernel, which is called kernel space. The lower 3GB byte is used by each process and is called user space.

To put it simply, Kernel Space is the Space where the system Kernel runs, and User Space is the Space where User programs run. They are isolated for security reasons.

System calls: user mode and kernel mode

Although there is a logical division between user space and kernel space, user space inevitably requires access to kernel resources, such as file operations, access to the network, and so on. In order to break through isolation restrictions, you need to use system calls to do this. System call is the only way for the user space to access the kernel space, which ensures that all resources are accessed under the control of the kernel, avoids unauthorized access to system resources by user programs, and improves the security and stability of the system.

Linux uses two levels of protection: level 0 for the system kernel and level 3 for user programs.

When a task (process) executes a system call and is executed in kernel code, the process is said to be in kernel mode. At this point the processor executes in the most privileged (level 0) kernel code. When a process is in kernel mode, kernel code executes using the current process’s kernel stack. Each process has its own kernel stack.

When a process executes the user’s own code, it is said to be in user mode (user mode). At this point the processor is running in user code at the lowest level of privilege (level 3).

The system call is mainly implemented by the following two functions:

Copy_from_user () // copy data from user space to kernel space copy_to_user() // copy data from kernel space to user spaceCopy the code

3.2 Traditional IPC Communication Principles in Linux

Understanding the above concepts, let’s take a look at the traditional IPC way, between the process is how to achieve communication.

The usual practice is that the data that the message sender will send is stored in an in-memory cache and entered into kernel mode through a system call. The kernel program then allocates memory in kernel space, creates a kernel cache, and calls copy_from_user() to copy data from the memory cache in user space to the kernel cache in kernel space. Similarly, the receiving process creates a memory cache in its user space when it receives data, and the kernel program then calls copy_to_user() to copy the data from the kernel cache to the receiving process’s memory cache. In this way, the data sender process and the data receiver process have completed a data transmission, which is said to have completed an interprocess communication. The diagram below:

There are two problems with this traditional IPC communication:

  1. Low performance, a data transfer needs to experience: memory cache –> kernel cache –> memory cache, need 2 data copies;
  2. Receive data buffer provided by the data receiving process, but the receiving process may not know how much space to store data to be passed, therefore can only be set up as large memory space or to call API to receive the size of the header to get the message body, the two is not a waste of space is a waste of time.

Principles of inter-process communication between Binders

After understanding Linux IPC concepts and communication principles, let’s formally introduce Binder IPC principles.

4.1 Dynamic kernel Loadable modules && Memory mapping

As mentioned earlier, cross-process communication is supported by kernel space. Traditional IPC mechanisms such as pipes and sockets are part of the kernel, so it is natural to implement inter-process communication through kernel support. But Binder is not part of the Kernel of a Linux system. What then? This is due to Linux’s dynamic Loadable Kernel Module (LKM) mechanism; A module is a stand-alone program that can be compiled independently, but cannot be run independently. It is linked to the kernel at runtime and runs as part of the kernel. In this way, The Android system can dynamically add a kernel module to run in the kernel space, and user processes can communicate with each other through this kernel module.

In Android, the kernel module that runs in the kernel space and is responsible for the communication between user processes through Binder is called a Binder Dirver.

How do user processes communicate with each other in The Android system through this kernel module (driven by Binder)? Is it possible to copy data from the sender process to the kernel cache, and then copy data from the kernel cache to the receiver process through two copies, like the traditional IPC mechanism mentioned earlier? Clearly not, or there would be no performance advantage of Binder.

This leads to another concept under Linux: memory mapping.

The memory mapping involved in the Binder IPC mechanism is implemented through mmap(), a method of memory mapping in the operating system. Memory mapping is simply a mapping of an area of memory in user space to kernel space. After the mapping relationship is established, the user’s modifications to this memory area can be directly reflected to the kernel space. In turn, changes made to this region in kernel space are reflected directly in user space.

Memory mapping reduces data copy times and enables efficient interaction between user space and kernel space. The modification of the two Spaces can be directly reflected in the mapped memory area, thus being sensed by the other space in time. Because of this, memory mapping provides support for interprocess communication.

4.2 Binder IPC Implementation Principles

Binder IPC is implemented based on memory mapping (MMAP), but mmap() is usually used on file systems with physical media.

For example, if you want to read data from disk into the user area of the process, you need to make two copies (disk –> kernel space –> user space). In this scenario, mmap() can be used to create a mapping between physical media and user space, reducing the number of data copies, and improving file reading efficiency by replacing I/O with memory read/write.

Binder does not have physical media, so the Binder driver uses mmap() not to create a mapping between physical media and user space, but to create cache space for data reception in kernel space.

A complete IPC communication process with Binder is usually as follows:

  1. First, the Binder driver creates a data receive cache in the kernel space.
  2. Then a kernel cache area is set up in the kernel space, and the mapping relationship between the kernel cache area and the data receiving cache area in the kernel, as well as the mapping relationship between the data receiving cache area and the user space address of the receiving process in the kernel is established.
  3. The sender process uses the system call copy_from_user() to copy the data to the kernel cache in the kernel. Since there is a memory mapping between the kernel cache and the user space of the receiving process, it is equivalent to sending the data to the user space of the receiving process, thus completing an inter-process communication.

The diagram below:

Binder communication model

After introducing the underlying communication principles of Binder IPC, let’s look at how the implementation layer is designed.

A complete interprocess communication must contain at least two processes. Usually, we call the communication parties as Client process and Server process respectively. Due to the existence of process isolation mechanism, the communication parties must use Binder to implement.

5.1 the Client/Server/ServiceManager/driver

As described earlier, Binder is based on a C/S architecture. It consists of a series of components, including the Client, Server, ServiceManager, and Binder drivers. Client, Server, and Service Manager run in user space, and Binder drivers run in kernel space. The Service Manager and Binder drivers are provided by the system, and the Client and Server are implemented by the application program. Client, Server, and ServiceManager access device file /dev/binder through system calls to open, mmap, and IOCtl, thereby achieving interaction with binder drivers to indirectly achieve cross-process communication.

Client, Server, ServiceManager, and Binder Drivers play a role in the communication process just like the relationship between servers (Server), clients (Client), DNS Server (ServiceManager), and routers (Binder drivers) in the Internet.

The usual way to access a web page is to enter an address in your browser, such as www.google.com, and then press enter. However, there is no way to directly find the server we want to access through the domain name address, so we need to access the DNS domain name server first. The DNS server stores the IP address 10.249.23.13 corresponding to www.google.com. Then through this IP address can be put to www.google.com corresponding server.

Android Binder design and implementationThis article describes the Client, Server, ServiceManager, and Binder drivers in detail. The following are excerpts:

Binder drivers Binder drivers, like routers, are the core of communication. The driver is responsible for the establishment of Binder communication between processes, the transmission of Binder between processes, the management of Binder reference count, the transmission and interaction of packets between processes and a series of low-level support. ServiceManager is similar to the real-name Binder. The ServiceManager converts the Binder name in the form of a character into a reference to the Binder in the Client. Enables clients to obtain references to Binder entities by Binder names. A Binder with a registered name is called a real name Binder, just like a website except it has an IP address and an ADDRESS of its own. The Server creates the Binder, gives it a character, readable and easy to remember name, and sends the Binder entity with its name as a packet to the ServiceManager through the Binder driver. Notifies the ServiceManager to register a Binder named “Three” on a Server. The driver creates an entity node in the kernel and a ServiceManager reference to the entity for the Binder crossing process boundaries, and packages the name and the new reference to the ServiceManager. The ServiceManger receives the data and fills in the lookup table with names and references. Careful readers may notice that ServierManager is one process and Server is another, and that registering a Server with the ServiceManager necessarily involves interprocess communication. The current implementation of interprocess communication is also using interprocess communication, which is like the premise that the egg can hatch the chicken is to find a chicken to lay the egg first! Binder’s clever implementation is to precreate a chicken to lay eggs. The ServiceManager communicates with other processes using Bidner. The ServiceManager is a Server and has its own Binder entity. All other processes are clients. This Binder reference is required to register, query, and retrieve the Binder. The ServiceManager provides a special Binder that has no name and does not require registration. When a process registers itself as a ServiceManager using the BINDER_SET_CONTEXT_MGR command, the Binder driver automatically creates Binder entities for it (this is the pre-built chicken). Second, the reference to this Binder entity is fixed to 0 in all clients and does not need to be obtained by other means. That is, a Server that wants to register its own Binder with The ServiceManager must communicate with the ServiceManager’s Binder through this zero reference. Like the Internet, reference 0 is like the address of a DOMAIN name server, which you must configure either dynamically or manually in advance. Note that the Client here is relative to the ServiceManager, a process or application may be a Server providing services, but for the ServiceManager it is still a Client. The Client obtains a reference to the real-name Binder. After the Server registers the Binder with the ServiceManager, the Client obtains a reference to the Binder by name. The Client also requests access to a Binder from the ServiceManager using the reserved reference number 0: I request access to the Binder reference named Zhang SAN. After receiving the request, the ServiceManager retrieves the Binder name from the request packet, finds the corresponding entry in the lookup table, and retrieves the corresponding Binder reference as a reply to the Client that initiates the request. From an object-oriented perspective, Binder entities in the Server now have two references: one in the ServiceManager and one in the Client that initiated the request. If more clients then request the Binder, there will be more references to the Binder in the system, just as there are multiple references to an object in Java.

5.2 Binder Communication Process

So far, we can summarize the Binder communication process:

  1. First, a process registers itself as a ServiceManager with the Binder driver using the BINDER_SET_CONTEXT_MGR command.
  2. The Server registers binders (Binder entities in the Server) with the ServiceManager through the driver to provide services externally. The driver creates an entity node in the kernel and a Reference to the entity by the ServiceManager for this Binder, packages the name and the newly created reference to the ServiceManager, and the ServiceManger fills it in the lookup table.
  3. The Client obtains a reference to the Binder entity from the ServiceManager with the help of the Binder driver through the name, and can communicate with the Server process through this reference.

We see a need for Binder driven access throughout the communication process. The following figure provides a more intuitive representation of the entire communication process (we omit the concept of Binder entities and references for the sake of further abstracting the communication process and rendering convenience) :

5.3 Agent Mode for Binder Communication

We’ve explained how clients and servers can communicate with binders across processes, but there’s still a problem. How does process A want an object from process B? After all, they are separate processes, so process A cannot directly use object from process B.

As we mentioned earlier, the Binder drivers are involved in cross-process communication, so as data flows through the Binder drivers, the driver performs a layer of transformation. When process A wants an object from process B, the driver does not return the object to process A. Instead, it returns an objectProxy that looks exactly like the object. The objectProxy has exactly the same methods as the object, but these methods don’t have the same capabilities as the methods of the object in process B. These methods only need to pass the request parameters to the driver. This is the same for process A as calling A method on object directly.

When the Binder driver receives A message from process A and finds that it is an objectProxy, it queries the form it maintains and finds that it is A proxy for process B’s object. It then tells B to call the method on object and asks B to send the result back to it. When the driver gets the return result of process B, it will forward to process A, A communication is completed.

5.4 Complete definition of Binder

Now we can make a more comprehensive definition of Binder:

  • From the perspective of interprocess communication, a Binder is a mechanism for interprocess communication.
  • From the perspective of a Server process, a Binder refers to a Binder entity object in a Server.
  • From the perspective of a Client process, a Binder refers to a proxy object for a Binder, which is a remote proxy for a Binder entity object
  • From a transport process perspective, a Binder is an object that can be transported across processes; The Binder driver does a little special work on objects that cross process boundaries, automatically converting between proxy objects and local objects.

6. Manually code cross-process calls

Usually when we do development, the most common way to implement interprocess communication is AIDL. When we define the AIDL file, the compiler generates code for IPC communication at compile time. With AIDL compiled code, we can further understand Binder IPC communication principle.

But compiler generated code is not developer friendly, either in terms of readability or understandability. For example, a Bookmanager.aidl file generates a BookManager.java file that contains a BookManager interface, a Stub static abstract class, and a Proxy static class. Proxy is a static inner class of Stub, which in turn is a static inner class of BookManager, which creates readability and understandability problems.

The design of Android actually makes sense, because putting BookManager, Stub, and Proxy in the same file prevents Stub and Proxy from having the same name when there are multiple AIDL files.

So for the sake of understanding, let’s manually write code to implement cross-process calls.

6.1 Job Description of Each Java class

Before formally coding for cross-process calls, let’s take a look at some of the classes used in the implementation. Understanding the responsibilities of these classes helps us better understand and implement cross-process communication.

  • IBinder: An IBinder is an interface that represents the ability to communicate across processes. Once this excuse is implemented, the object can be transferred across processes.

  • IInterface: IInterface represents the capabilities of the Server process object (which methods it provides, in fact, correspond to the interface defined in the AIDL file)

  • Binder: A Binder class in the Java layer that represents Binder local objects. The BinderProxy class is an inner class of the Binder class that represents the local proxy for the remote process’s Binder objects. Both classes inherit from IBinder and thus have the ability to transport across processes; In fact, the Binder driver automatically converts the two objects across the process.

  • Stub: With AIDL, the compiler generates a static inner class called Stub. This class inherits from Binder, indicating that it is a Binder local object that implements the IInterface interface, indicating that it has the capabilities promised by the Server to the Client. Stub is an abstract class, and the implementation of the concrete IInterface needs to be implemented by the developer.

6.2 Implementation process explanation

In this case, RemoteService serves as the server process. ClientActivity acts as the client process and uses the services provided by RemoteService. The diagram below:

So what are the capabilities of the server process? What services can be provided to the client? Remember the IInterface, which represents the specific capabilities of the server process. So we need to define a BookManager interface, which inherits from IIterface, to indicate what capabilities the server has.

/** * This class defines what the server RemoteService is capable of */
public interface BookManager extends IInterface {

    void addBook(Book book) throws RemoteException;
}
Copy the code

It is not enough to define what capabilities the server has. Since it is a cross-process call, then we need to implement a cross-process call object Stub. Stub inherits from the Binder, indicating that it is a Binder local object. Implement the IInterface interface, indicating that the Server has the capability promised to the Client; Stub is an abstract class, and the specific implementation of the IInterface needs to be implemented by the caller.

public abstract class Stub extends Binder implements BookManager {...public static BookManager asInterface(IBinder binder) {
        if (binder == null)
            return null;
        IInterface iin = binder.queryLocalInterface(DESCRIPTOR);
        if(iin ! =null && iin instanceof BookManager)
            return (BookManager) iin;
        return newProxy(binder); }...@Override
    protected boolean onTransact(int code, Parcel data, Parcel reply, int flags) throws RemoteException {
        switch (code) {

            case INTERFACE_TRANSACTION:
                reply.writeString(DESCRIPTOR);
                return true;

            case TRANSAVTION_addBook:
                data.enforceInterface(DESCRIPTOR);
                Book arg0 = null;
                if(data.readInt() ! =0) {
                    arg0 = Book.CREATOR.createFromParcel(data);
                }
                this.addBook(arg0);
                reply.writeNoException();
                return true;

        }
        return super.onTransact(code, data, reply, flags); }... }Copy the code

The Stub class focuses on asInterface and onTransact.

When a Client calls bindService, it creates a ServiceConnection object as an input parameter. In the ServiceConnection’s callback method onServiceConnected, you get the BookManager object through this asInterface(IBinder Binder), The IBinder type of the binder is driven to our, as you see, in the code method to call binder. The queryLocalInterface () to find the binder local objects, If found, the Client and Server are in the same process, and the binder itself is a binder local object and can be used directly. Otherwise, the binder is a remote object, that is, a BinderProxy. Therefore, we need to create a Proxy object to implement remote access through this Proxy object.

The next step is to implement the Proxy class, which naturally implements the BookManager interface.

public class Proxy implements BookManager {...public Proxy(IBinder remote) {
        this.remote = remote;
    }

    @Override
    public void addBook(Book book) throws RemoteException {

        Parcel data = Parcel.obtain();
        Parcel replay = Parcel.obtain();
        try {
            data.writeInterfaceToken(DESCRIPTOR);
            if(book ! =null) {
                data.writeInt(1);
                book.writeToParcel(data, 0);
            } else {
                data.writeInt(0);
            }
            remote.transact(Stub.TRANSAVTION_addBook, data, replay, 0);
            replay.readException();
        } finally{ replay.recycle(); data.recycle(); }}... }Copy the code

Let’s look at the implementation of addBook(); In the Stub class, addBook(Book Book) is an abstract method that the Client inherits and implements.

  • If the Client and Server are in the same process, then this method is called directly.
  • If the Client wants to call the Server’s methods remotely, it needs to do so through the Binder Proxy, which is the Proxy above.

The data is first serialized by Parcel in the addBook() method in the Proxy, followed by a call to remote.transact(). As mentioned earlier, a Proxy is created in a Stub asInterface. Going to the step of creating a Proxy means that the input to the Proxy constructor is a BinderProxy. In this case, remote is a BinderProxy object. Finally, through a series of function calls, the Client process gets into kernel mode through system calls, and the Client process thread that executes addBook() hangs and waits to return; After the driver completes a series of operations, it wakes up the Server process and calls onTransact() on the Server process’s local object. Finally, in the Stub class onTransact(), onTransact() calls functions based on function numbers (the Stub class defines a number for each function in the BookManager interface, but we simplified it in the source code above; In cross-process calls, instead of passing a function, pass a number to indicate which function to call); In our example, addBook() of the Binder local object is called and the result is returned to the driver, who wakes up the thread in the Client process that has just been suspended and returns the result.

This completes a cross-process call.

I put the complete code on GitHub, interested partners can check it out. Source address: github.com/BaronZ88/He…

Finally, it is recommended that you implement the communication between Client and Server processes by hand without AIDL to deepen the understanding of Binder’s communication process.

Due to the limitation of individual ability level, there will inevitably be mistakes in the article. If you find the shortcomings of the article, welcome to communicate with me.

Many articles, books and source codes have been referenced during the writing process of this article. Many descriptions and pictures have been borrowed from the following articles. Thanks for your selfless sharing!

References are as follows:

  • Android Binder design and Implementation – Design
  • Android Interprocess Communication (IPC) Mechanism Binder brief Introduction and Learning Plan, Android Source Code Scenario Analysis
  • Binder Learning Guide
  • Binder Series
  • Binder describes how Binder communicates across processes
  • Android’s Binder mechanism is simple
  • User space versus kernel space
  • Carefully analyze MMAP: what is it, why and how to use it

If you like my posts, follow my BaronTalk, Zhihu.com column, or add a Star on GitHub.

  • Wechat official account: BaronTalk
  • Zhihu column: zhuanlan.zhihu.com/baron
  • GitHub：github.com/BaronZ88
  • Personal blog: baronzhang.com