The Master said, “The people may command them, but not know them.” The Analects of Confucius: Tabor

A hundred blog series. This paper is: the v24. Xx HongMeng kernel source code analysis concept (process) | process in which resources management

Process management:

  • V02. Xx HongMeng kernel source code analysis (process management) | who in the management of core resources
  • V24. Xx HongMeng kernel source code analysis concept (process) | process in which resources management
  • V45. Xx HongMeng kernel source code analysis (Fork) | one call, return twice
  • V46. Xx HongMeng kernel source code analysis (special) | mouse son can make hole
  • V47. Xx HongMeng kernel source code analysis (process recycling) | how to entrust an orphan to ancestor dying
  • V48. Xx HongMeng kernel source code analysis (signal) | the year lead half hundred, still energetic
  • V49. Xx HongMeng kernel source code analysis (signal) | who let CPU run for four times in the stack
  • V71. Xx HongMeng kernel source code analysis (Shell editor) | two tasks, three stages
  • V72. Xx HongMeng kernel source code analysis (Shell) | watching the kernel of the application window

This section explains the process

Before reading this article, it is recommended to read hongmeng kernel source code analysis (total directory) scheduling story, which has the metaphor of the process life scene.

Official basic concept

  • From a system perspective, processes are resource management units. Processes can use or wait for system resources such as CPU, use memory space, and run independently of other processes.

  • The process module of hongmeng kernel can provide users with multiple processes, realize the switch and communication between processes, and help users to manage business procedures. This allows users to focus more on the implementation of business functions.

  • The process in hongmeng kernel adopts preemptive scheduling mechanism and supports time slice rotation scheduling and FIFO scheduling mechanism.

  • Hongmeng kernel process has a total of 32 priorities (0-31), user process can configure 22 priorities (10-31), the highest priority is 10, the lowest priority is 31.

  • A process with a higher priority can preempt a process with a lower priority. A process with a lower priority can be scheduled only after the process with a higher priority blocks or finishes.

  • Each user-mode process has its own independent process space, invisible to each other, to achieve inter-process isolation.

Official Concept interpretation

The most important sentence in the official document is that a process is a resource management unit, which manages resources. What are resources? Memory, tasks, files, semaphores, and so on are all resources. In the story, I made an image of the process (director), responsible for the performance of the program (task), responsible for coordinating the various resources needed when the program runs. So that the program can be efficiently and smoothly completed.

Hongmeng kernel source code analysis positioning for deep digging kernel foundation, build the bottom network map. The original LosProcessCB is as follows, here we break it down one by one to see what it looks like.

ProcessCB mami

typedef struct ProcessCB {
    CHAR                 processName[OS_PCB_NAME_LEN]; /**< Process name */ // Process name
    UINT32               processID;                    /**< process ID = leader thread ID */ // process ID, assigned by the process pool, range [0,64]
    UINT16               processStatus;                /**< [15:4] process Status; [3:0] The number of threads currently running in the process */// The design here is clever. A 16 represents two logical levels of quantities and states. Thumbs up!
    UINT16               priority;                     /**< process priority */ // Process priority
    UINT16               policy;                       /**< process policy */ // Process scheduling mode, preemptive by default
    UINT16               timeSlice;                    /**< Remaining time slice */// Process time slice, default 2 tick
    UINT16               consoleID;                    /**< The console id of task belongs */// The console ID of the task
    UINT16               processMode;                  /**< Kernel Mode:0; User Mode:1; * / // Mode is specified as kernel or user process
    UINT32               parentProcessID;              /**< Parent process ID */ // ID of the parent process
    UINT32               exitCode;                     /**< process exit status */ // Process exit status code
    LOS_DL_LIST          pendList;                     /**< Block list to which the process belongs */ // The blocking list to which the process belongs. If the lock fails, this node is attached to the list
    LOS_DL_LIST          childrenList;                 /**< my children process list */ // The child processes hang here, forming a double-loop linked list
    LOS_DL_LIST          exitChildList;                /**< my exit children process list */ // Those who want to quit the child process and hang here, gray hair to see black hair.
    LOS_DL_LIST          siblingList;                  /**< linkage in my parent's children list */ // All 56 races are from the same parent process.
    ProcessGroup         *group;                       /**< Process group to which a process belongs */ // Owning process group
    LOS_DL_LIST          subordinateGroupList;         /**< linkage in my group list */ // What team processes are available when the process is the group leader
    UINT32               threadGroupID;                /**< thread group, is the main thread ID of the process */ // Which thread group is the main thread ID of the process
    UINT32               threadScheduleMap;            /**< The scheduling bitmap table for the thread group of the process */ // Process scheduling bitmap for each thread
    LOS_DL_LIST          threadSiblingList;            /**< List of threads under this process */// List of threads (tasks) for the process
    LOS_DL_LIST          threadPriQueueList[OS_PRIORITY_QUEUE_NUM]; /**< The process's thread group schedules the priority hash table */ // The thread group scheduling priority hash table of the process
    volatile UINT32      threadNumber; /**< Number of threads alive under this process */ // The number of active threads in this process
    UINT32               threadCount;  /**< Total number of threads created under this process */ // The total number of threads created under this process
    LOS_DL_LIST          waitList;     /**< The process holds the waitLits to support wait/waitpid */// Processes hold wait lists to support wait/ waitPID
#if (LOSCFG_KERNEL_SMP == YES)
    UINT32               timerCpu;     /**< CPU core number of this task is delayed or pended */// Count the time each thread is delayed or blocked
#endif
    UINTPTR              sigHandler;   /**< signal handler */ // Signal processing functions, such as SIGSYS
    sigset_t             sigShare;     /**< signal share bit */ // Signals share bits
#if (LOSCFG_KERNEL_LITEIPC == YES)
    ProcIpcInfo         ipcInfo;       /**< memory pool for lite ipc */ // A virtual device file system for interprocess communication. The device mount point is /dev/lite_ipc
#endif
    LosVmSpace          *vmSpace;       /**< VMM space for processes */ // Virtual space, a data structure that describes a process's virtual memory. Linux calls it a memory descriptor
#ifdef LOSCFG_FS_VFS
    struct files_struct *files;        /**< Files held by the process */ // All files held by the process are called the file manager of the process
#endif // Each process has its own file manager, which records operations on files. Note: A file can be manipulated by more than one process
    timer_t             timerID;       /**< iTimer */

#ifdef LOSCFG_SECURITY_CAPABILITY // Security capability
    User                *user;  // Process owner
    UINT32              capability; // The security capability range corresponds to CAP_SETGID
#endif
#ifdef LOSCFG_SECURITY_VID
    TimerIdMap          timerIdMap;
#endif
#ifdef LOSCFG_DRIVERS_TZDRIVER
    struct file         *execFile;     /**< Exec bin of the process */
#endif
    mode_t umask;
} LosProcessCB;
Copy the code

The structure is still quite complex, although each one is annotated, but still not clear enough, not modular. Here it is broken down into the following six pieces:

The first block: the relationship with tasks (threads)

    UINT32               threadGroupID;                /**< thread group, is the main thread ID of the process */ // Which thread group is the main thread ID of the process
    UINT32               threadScheduleMap;            /**< The scheduling bitmap table for the thread group of the process */ // Process scheduling bitmap for each thread
    LOS_DL_LIST          threadSiblingList;            /**< List of threads under this process */// List of threads (tasks) for the process
    LOS_DL_LIST          threadPriQueueList[OS_PRIORITY_QUEUE_NUM]; /**< The process's thread group schedules the priority hash table */ // The thread group scheduling priority hash table of the process
    volatile UINT32      threadNumber; /**< Number of threads alive under this process */ // The number of active threads in this process
    UINT32               threadCount;  /**< Total number of threads created under this process */ // The total number of threads created under this process
    LOS_DL_LIST          waitList;     /**< The process holds the waitLits to support wait/waitpid */// Processes hold wait lists to support wait/ waitPID
Copy the code

The relationship between a process and a thread is 1:N. A process can have multiple tasks, but a task cannot belong to multiple processes. A task is a thread, a scheduling unit of the CPU. The concept of threads in the Hongmeng kernel source code analysis (total directory) in the thread section has a detailed introduction, can be viewed by yourself. Tasks are managed as resources by processes that provide memory support, file support, and device support for tasks.

How does a process manage threads and synchronize thread state?

  • 1. When the process loads, it finds the main function to create the first thread, usually the main thread. The main function is the entry function, where everything starts.

  • 2. During execution, a new thread is created according to the code (for example, when a new thread is encountered in Java), which is essentially the same as the thread created by the main function, but the entry function is changed into run() to participate in the unified scheduling.

  • The relationship between a thread and a thread can be detached or join. Joins are when one thread can manipulate another thread (including reclaiming resources and being killed by the other thread).

  • 4. After the main thread or all threads of a process finish running, the process changes to zombie state. Generally, the process can die only after all threads end.

  • 5. After the process is created, it enters the ready state. When a process switchover occurs, the process with the highest priority in the ready list is executed and enters the running state. If there is no other thread in the ready state, the process is deleted from the ready list and only in the running state. If there are other threads in the ready state, the process is still in the ready queue. At this time, the ready state and running state of the process coexist. Note here that processes can allow multiple states to coexist! State coexistence naturally brings to mind bitmap management, which is covered in detail in this series.

  • 6. If all threads in a process are in the blocked state, the process synchronously enters the blocked state when the last thread becomes blocked, and then the process switchover occurs.

  • 7. When any thread in the blocked process restores to the ready state, the process is added to the ready queue and synchronized to the ready state. If process switchover occurs at this time, the state of the process changes from the ready state to the running state

  • 8. When the last ready thread in a process is blocked, the process is removed from the ready list and changes from ready to blocked.

  • 9. There are two situations when a process changes from a running state to a ready state:

    • After a process with a higher priority is created or restored, process scheduling occurs. At this moment, the process with the highest priority in the ready list becomes running, and the previously running process changes from running state to ready state.

    • If a process with SCHED_RR is preempt and another process with the same priority is in the ready state, it goes from running to ready and another process with the same priority goes from ready to running when its time slices run out.

Second block: Relationships with other processes

    CHAR                 processName[OS_PCB_NAME_LEN]; /**< Process name */ // Process name
    UINT32               processID;                    /**< process ID = leader thread ID */ // process ID, assigned by the process pool, range [0,64]
    UINT16               processStatus;                /**< [15:4] process Status; [3:0] The number of threads currently running in the process */// The design here is clever. A 16 represents two logical levels of quantities and states. Thumbs up!
    UINT16               priority;                     /**< process priority */ // Process priority
    UINT16               policy;                       /**< process policy */ // Process scheduling mode, preemptive by default
    UINT16               timeSlice;                    /**< Remaining time slice */// Process time slice, default 2 tick
    UINT16               consoleID;                    /**< The console id of task belongs */// The console ID of the task
    UINT16               processMode;                  /**< Kernel Mode:0; User Mode:1; * / // Mode is specified as kernel or user process
    UINT32               parentProcessID;              /**< Parent process ID */ // ID of the parent process
    UINT32               exitCode;                     /**< process exit status */ // Process exit status code
    LOS_DL_LIST          pendList;                     /**< Block list to which the process belongs */ // The blocking list to which the process belongs. If the lock fails, this node is attached to the list
    LOS_DL_LIST          childrenList;                 /**< my children process list */ // The child processes hang here, forming a double-loop linked list
    LOS_DL_LIST          exitChildList;                /**< my exit children process list */ // Those who want to quit the child process and hang here, gray hair to see black hair.
    LOS_DL_LIST          siblingList;                  /**< linkage in my parent's children list */ // All 56 races are from the same parent process.
    #if (LOSCFG_KERNEL_LITEIPC == YES)
    ProcIpcInfo         ipcInfo;       /**< memory pool for lite ipc */ // A virtual device file system for interprocess communication. The device mount point is /dev/lite_ipc
    #endif
Copy the code

Processes are family-managed. Kernel-mode processes and user-mode processes have their own root ancestors, which are created when the kernel is initialized. They are the no. 1 (user-process ancestor) and no. 2 (kernel-process ancestor) processes. The process is genetically determined as soon as you’re born, and your genes determine your permissions depending on who your father is, who your brothers and sisters are, and just like people, you can’t choose to be born. But processes can have their own children and grandchildren, descended from your lineage, much like humans. The result is a tree structure where each process finds its place. Process management follows the following principles:

  • 1. When the process exits, the process resources are released. However, the pid resources must be reclaimed by the parent process in wait/ WaitPID mode or when the parent process exits.

  • 2. The death of a child process should be notified to the parent process, so that the parent process can erase its traces in the family tree. If the death of a bad child process in some abnormal circumstances is not notified to the parent process, the system will also have a scheduled task to detect and reclaim its resources.

  • 3. After a process is created, it can only operate resources in its own process space but cannot operate resources (except shared resources) of other processes.

  • 4. There are a variety of communication methods between processes, such as events, signals, message queues, pipes, etc. Liteipc is a file-based communication method between processes, which is characterized by the transfer of information can be very large.

  • 5. A process with a higher priority can preempt a process with a lower priority. A process with a lower priority can be scheduled only after the process with a higher priority blocks or finishes.

Third block: The five states of the process

  • Init: The process is being created.

  • Ready: The process is in the Ready list, waiting for CPU scheduling.

  • Running: The process is Running.

  • Blocked (Pend) : The process is blocked and suspended. A process is blocked and suspended when all threads in the process are blocked.

  • Zombies: The process stops running and waits for the parent process to reclaim its control block resources.

Fourth block: the relationship with memory

    LosVmSpace          *vmSpace;       /**< VMM space for processes */ // Virtual space, a data structure that describes a process's virtual memory. Linux calls it a memory descriptor
Copy the code
  • Each user-mode process has its own independent process space, invisible to each other, to achieve isolation between processes. Independent process space means that each process has to map its own virtual memory and physical memory. And save the mapping area in its own process space. In addition, the process of the code area, data area, stack area, mapping area are stored in their own space, but the kernel-mode process space is shared, only a mapping.
  • Specific into the Hongmeng kernel source code analysis (total directory) view the memory section. The concept and realization of virtual memory, physical memory, linear address, mapping relationship, shared memory, allocation recycling, page replacement are introduced in detail.

Fifth block: the relationship with documents

#ifdef LOSCFG_FS_VFS
    struct files_struct *files;        /**< Files held by the process */ // All files held by the process are called the file manager of the process
#endif // Each process has its own file manager, which records operations on files. Note: A file can be manipulated by more than one process
Copy the code

Files_struct is the file manager of a process, and files are also a very complex chunk. There will be a series of articles on file system implementation. The main context for understanding a file system is:

  • 1. A real physical file (inode) that can be opened by multiple processes at the same time and has process-independent file descriptors. ProcessFD is followed by SystemFD.

  • System file descriptors (0-stdin, 1-stdout, 2-stderr) are used by the kernel by default. The first three file descriptors of any process are (stdin, stdout, stderr), which are opened by default.

  • 3. File mapping Similar to memory mapping, each process needs to map the same file separately. Page_mapping records the mapping relationship, and page cache provides the actual memory location of the file.

  • 4. Memory <-> file replacement by page unit (4K), the process cannot directly operate on the hard disk file, must be through the page cache. Classic concepts such as copy-on-write (COW) technology are involved. More on that later.

The sixth block: auxiliary tools

#if (LOSCFG_KERNEL_SMP == YES)
    UINT32               timerCpu;     /**< CPU core number of this task is delayed or pended */// Count the time each thread is delayed or blocked
#endif
#ifdef LOSCFG_SECURITY_CAPABILITY // Security capability
    User                *user;  // Process owner
    UINT32              capability; // The security capability range corresponds to CAP_SETGID
#endif
#ifdef LOSCFG_SECURITY_VID
    TimerIdMap          timerIdMap;
#endif
#ifdef LOSCFG_DRIVERS_TZDRIVER
    struct file         *execFile;     /**< Exec bin of the process */
#endif
Copy the code

The rest are some security, statistical capabilities.

This is the full picture of the process, and it will be much clearer if you look at it in the core!

Intensive reading of the kernel source code

Four code stores synchronous annotation kernel source code, >> view the Gitee repository

Analysis of 100 blogs. Dig deep into the core

Add comments to hongmeng kernel source code process, sort out the following article. Content based on the source code, often in life scene analogy as much as possible into the kernel knowledge of a scene, with a pictorial sense, easy to understand memory. It’s important to speak in a way that others can understand! The 100 blogs are by no means a bunch of ridiculously difficult concepts being put forward by Baidu. That’s not interesting. More hope to make the kernel become lifelike, feel more intimate. It’s hard, it’s hard, but there’s no turning back. 😛 and code bugs need to be constantly debug, there will be many mistakes and omissions in the article and annotation content, please forgive, but will be repeatedly amended, continuous update. Xx represents the number of modifications, refined, concise and comprehensive, and strive to create high-quality content.

Compile build The fundamental tools Loading operation Process management
Compile environment

The build process

Environment script

Build tools

Designed.the gn application

Ninja ninja

 Two-way linked list

Bitmap management

In the stack way

The timer

Atomic operation

Time management

 The ELF format

The ELF parsing

Static link

relocation

Process image

 Process management

Process concept

Fork

Special process

Process recycling

Signal production

Signal consumption

Shell editor

Shell parsing
Process of communication Memory management Ins and outs Task management
spinlocks

The mutex

Process of communication

A semaphore

Incident control

The message queue

 Memory allocation

Memory management

Memory assembly

The memory mapping

Rules of memory

Physical memory

 Total directory

Scheduling the story

Main memory slave

The source code comments

Source structure

Static site

 The clock task

Task scheduling

Task management

The scheduling queue

Scheduling mechanism

Thread concept

Concurrent parallel

The system calls

Task switching
The file system Hardware architecture
File concept

The file system

The index node

Mount the directory

Root file system

Character device

VFS

File handle

Pipeline file

 Compilation basis

Assembly and the cords

Working mode

register

Anomaly over

Assembly summary

Interrupt switch

Interrupt concept

Interrupt management

HongMeng station | into a little bit every day, the original is not easy, welcome to reprint, please indicate the source.