The middle bow asks for mercy. The Master said, “When you go out, be as if you saw a great guest, and make the people as if they were offering a great sacrifice. Do unto others as you would have them do unto you. No complaint at home, no complaint at home.” Zhong Gong yue: “yong although not sensitive, please matter si language yi.” The Analects of Confucius: Yan Yuan

A hundred blog series. This is:

  • [v59. Xx HongMeng kernel source code analysis (build tools) | suitable melon touch cane debugging HongMeng build process

Compilation and construction related articles are:

  • V50. Xx HongMeng kernel source code analysis (compile environment) | compiler HongMeng drop pit prevention guide
  • V57. Xx HongMeng kernel source code analysis (compilation) | simple case at compilation process
  • Script v58. Xx HongMeng kernel source code analysis (environment) | compiler HongMeng so simple
  • V59. Xx HongMeng kernel source code analysis, build tools | suitable melon touch cane debugging HongMeng build process
  • Its v60. Xx HongMeng kernel source code analysis (designed.the gn) | designed.the gn syntax and use in HongMeng
  • V61. Xx HongMeng kernel source code analysis (ninja ninja) | ninja can upset

The need to build

  • Front-end development has build tools:Grunt,Gulp,Webpack
  • Background development has build tools:Maven,Ant,Gradle

The build tool is very important. It describes the rules of how to compile, connect, and package the entire project, including: which source files in the project need to be compiled and how to compile, which library files need to be created and how to create them, and how to output the desired files.

The compiler and build tool of the Hong Kong Light Kernel (L1/ LitEOS) is HB, which is short for Ohos-Build, and OHOs is short for OpenHarmony OS.

hb | ohos-build

Hb is installed with the following command and is a build tool written in Python.

python3 -m pip install --user ohos-build
Copy the code

The source code is in the./build/lite directory, meaning as follows:

Build/lite ├ ─ ─ components # component description file ├ ─ ─ figures # the pictures in the readme ├ ─ ─ hb # hb PIP installation package source │ ├ ─ ─ the build # hb build command implementation │ ├ ─ ─ the clean │ ├─ Common # Hb Clean Provide Device, the Config, Product, utils class │ ├ ─ ─ CTS # hb CTS command implementation │ ├ ─ ─ deps # hb deps command implementation │ ├ ─ ─ env # hb env command implementation │ ├ ─ ─ set # ├─ Make_rootfs # ├─ Config # └ │ ├─ Component # ├─ ├─ kernel # ├─ ├─ Set ├─ make_rootfs # ├─ config # ├─ Component # ├─ │ ├─ kernel # ├─ ├─ Platform # ├─ Testfwk # ├─ toolchain ├─ compiler path, compiler options, link options, etcCopy the code

Building of

The Buildsystem consists of python, GN, Ninja, and Makefile components, each with its own mission to do what it is best at.

  • Python: the glue language, best known for parameters, environment variables, and file manipulation, is used to prepare for compilation and collect command parameters for the GN. How about gn instead of Python? That’s fine, but it’s a hassle. For example, using the following command line, you can also generate a.ninja file, which will have the same effect. But compared to just usinghb buildWhich smells better,hb buildIt also generates the following blob, but that’s python’s strong suit. 
    / home/tools/designed.the gn gen/home/openharmony/code - v1.1.1 - LTS/out/hispark_aries/ipcamera_hispark_aries \ - the root = / home/openharmony/code - v1.1.1 - LTS \ - dotfile = / home/openharmony/code - v1.1.1 - LTS/build/lite /. Designed.the gn \ --script-executable=python3 \ '--args=ohos_build_type="debug" ohos_build_compiler_specified="clang" ohos_build_compiler_dir="/home/tools/llvm" Product_path = "/ home/openharmony/code - v1.1.1 - LTS/vendor/hisilicon/hispark_aries" Device_path = "/ home/openharmony/code - v1.1.1 - LTS/device/hisilicon hispark_aries/sdk_liteos" ohos_kernel_type = "liteos_a" enable_ohos_appexecfwk_feature_ability = false ohos_full_compile=true'Copy the code

    The picture shows the result of gn Gen directly bypassing the HB Python part:

  • Gn: A high-level build language,gn and Ninja are similar to C and assembly language, with cmake being used to generate ninja files. That’s fine, but it’s more trouble. It’s like writing a hongmeng system all in exchange, theoretically possible, but who would do that?
  • Ninja: An assembly language similar to the build world, with make as the opposite of it, using the compiler clang and linker LD.
  • -Makefile: Some of our modules are still compiled by make. I heard that we will use Ninja in the future. I don’t think we will see make files anymore.

How to debug HB

It is recommended to use VScode to debug HB. Click create a launch.json file in the debug panel to create a debug file.

{" version ":" 0.2.0, "" configurations: [{/ / hb set" name ":" hb set ", "type" : "python", "request" : "launch", "program": "./build/lite/hb/__main__.py", "console": "integratedTerminal", "args": ["set"], "stopOnEntry": true }, {//hb build "name": "hb build debug", "type": "python", "request": "launch", "program": "./build/lite/hb/__main__.py", "console": "integratedTerminal", "args": ["build"], "stopOnEntry": true }, {//hb clean "name": "hb clean", "type": "python", "request": "launch", "program": "./build/lite/hb/__main__.py", "console": "integratedTerminal", "args": ["clean"], "stopOnEntry": true }, ] }Copy the code

The build process

As shown in the build flowchart, there are two main steps: setup and compile:

This adjustable hb in trying to set | hb build two commands

Hb set | select project

Source code can be found in:. / build/lite/hb/set/set. Py hb set of execution process is roughly like this:

  • You can run HB Set under any directory, which attempts to read OCOMPANIES Config. json configuration files from the current directory, with no path that would ask you to enter code

    [OHOS INFO] Input code path: 
Copy the code

    The source code root directory, companies config.json, companies that generate configuration files, companies with fixed items managed by the Config class.

  • You can debug the set command with a breakpoint at the following location to track the entire process.

    def exec_command(args):
    if args.root_path is not None:  
        return set_root_path(root_path=args.root_path)

    if args.product:
        return set_product()

    return set_root_path() == 0 and set_product() == 0
Copy the code

    The picture shows the breakpoint debugging site

  • The resulting configuration file is as follows:

    {"root_path": "/home/ openHarmony/code-v1.1.1-lts ", "board": "hispark_aries", "kernel": "liteos_A ", "product": "ipcamera_hispark_aries", "product_path": "/ home/openharmony/code - v1.1.1 - LTS/vendor/hisilicon/hispark_aries", "device_path" : "/ home/openharmony/code - v1.1.1 - LTS/device/hisilicon/hispark_aries/sdk_liteos", "patch_cache" : null}Copy the code

    These paths pave the way for subsequent HB builds.

Hb build | compiler project

/build/lite/hb/build/*.py This article has annotated the important code in the compilation tool. Can also go to weharmony | annotation HongMeng compile tools check code comments for the project.

The overall steps are divided into two steps:

  • Call gn_buildgn genGenerate a *. Ninja file
  • Call ninja_buildninja -w dupbuild=warn -CGenerate *.o *.so *.bin and other final files

Gn_build information about THE GN can be found in the GN Reference manual. How gn was generated. Ninja file, the syntax and its use in Hongmeng will be described in detail in the following section.

Def gn_build(self, cmd_args): # Gn CMD init and makedirs(self.config. Out_path) # Gn CMD init and makedirs(self.config Gn gn_path = self.config.gn_path gn_args = cmd_args. Get ('gn', []) gn_cmd = [gn_path,#gn 'gen', self.config.out_path, #/home/openharmony/out/hispark_aries/ipcamera_hispark_aries '--root={}'.format(self.config.root_path), # project for example: the root of/home/openharmony '-- dotfile = {} /. Designed.the gn'. The format (self. Config. Build_path), # / home/openharmony/build/lite /. Designed.the gn - > root = "//build/lite" f'--script-executable={sys.executable}',#python3 '--args={}'.format(" ".join(self._args_list))] + gn_args  # ohos_build_type="debug" # ohos_build_compiler_specified="clang" # ohos_build_compiler_dir="/root/llvm" # product_path="/home/openharmony/vendor/hisilicon/hispark_aries" # device_path="/home/openharmony/device/hisilicon/hispark_aries/sdk_liteos" # ohos_kernel_type="liteos_a" # Enable_ocompanies AppEXECFWK_Feature_ability = false # OCOMPANIES FULL_compile =true' # These parameters will also be companies I can query in the ARGs.gn file following exec_Command Exec_command (gn_cmd, log_path=self.config.log_path)# run gn gen.. /out/hispark_aries/ipcamera_hispark_aries to generate the following file generated by the # obj subcompilation Ninja file directory, e.g. Obj /base/global/resmgr_lite/frameworks/resmgr_lite/global_resmgr. ohos_build_type="debug" ... # build.ninja sub-build items such as: build AA: phony dev_tools/bin/aa # build.ninja. /.. /.. /base/global/resmgr_lite/frameworks/resmgr_lite/ build.gn # toolchain ninja toolchain has put various BUILD/link rules rule CXX rule alinkCopy the code

Ninja_build More information about Ninja can be found in the Ninja Reference Manual. There will be more details on how Ninja works and its syntax and use in Hongmeng.

Def ninja_build(self, cmd_args): ninja_path = self.config.ninja_path ninja_args = cmd_args.get('ninja', []) ninja_cmd = [ninja_path, '-w', 'dupbuild=warn', '-C', self.config.out_path] + ninja_args # ninja -w dupbuild=warn -C / home/openharmony/out/hispark_aries/ipcamera_hispark_aries # will read designed.the gn generated files, compiling the second step, finally compiled into. O. bin file exec_command(ninja_cmd, log_path=self.config.log_path, Log_filter =True) # Companies file ocompanies image.bin bin build. Ninja Config etc LIbs OBj server.map test_info userfs #OHOS_Image OHOS_Image.map bm_tool.map build.ninja.d data foundation.map liteos.bin rootfs.tar suites toggleButtonTest.map userfs_jffs2.img #OHOS_Image.asm args.gn build.log bundle_daemon_tool.map dev_tools gen media_server.map rootfs_jffs2.img test toolchain.ninja vendorCopy the code

exec_command | utils.py

Gn_build and NINJa_build both end up calling exec_command to execute commands. Exec_command is a general method seen in listing 1 Build/lite/hb/common/utils py, debugging Suggestions here when breaking point, suitable melon touch rattan, tracking the implementation details of the related function.

def exec_command(cmd, log_path='out/build.log', **kwargs): useful_info_pattern = re.compile(r'\[\d+/\d+\].+') is_log_filter = kwargs.pop('log_filter', False) with open(log_path, 'at', encoding='utf-8') as log_file: process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='utf-8', **kwargs) for line in iter(process.stdout.readline, ''): if is_log_filter: info = re.findall(useful_info_pattern, line) if len(info): hb_info(info[0]) else: hb_info(line) log_file.write(line) process.wait() ret_code = process.returncode if ret_code ! = 0: with open(log_path, 'at', encoding='utf-8') as log_file: for line in iter(process.stderr.readline, ''): if 'ninja: warning' in line: log_file.write(line) continue hb_error(line) log_file.write(line) if is_log_filter: get_failed_log(log_path) hb_error('you can check build log in {}'.format(log_path)) if isinstance(cmd, list): cmd = ' '.join(cmd) raise Exception("{} failed, return code is {}".format(cmd, ret_code))Copy the code

The picture shows the breakpoint debugging site

Intensive reading of the kernel source code

Four code stores synchronous annotation kernel source code, >> view the Gitee repository

Analysis of 100 blogs. Dig deep into the core

Add comments to hongmeng kernel source code process, sort out the following article. Content based on the source code, often in life scene analogy as much as possible into the kernel knowledge of a scene, with a pictorial sense, easy to understand memory. It’s important to speak in a way that others can understand! The 100 blogs are by no means a bunch of ridiculously difficult concepts being put forward by Baidu. That’s not interesting. More hope to make the kernel become lifelike, feel more intimate. It’s hard, it’s hard, but there’s no turning back. 😛 and code bugs need to be constantly debug, there will be many mistakes and omissions in the article and annotation content, please forgive, but will be repeatedly amended, continuous update. Xx represents the number of modifications, refined, concise and comprehensive, and strive to create high-quality content.

Compile build The fundamental tools Loading operation Process management
Compile environment

The build process

Environment script

Build tools

Designed.the gn application

Ninja ninja

 Two-way linked list

Bitmap management

In the stack way

The timer

Atomic operation

Time management

 The ELF format

The ELF parsing

Static link

relocation

Process image

 Process management

Process concept

Fork

Special process

Process recycling

Signal production

Signal consumption

Shell editor

Shell parsing
Process of communication Memory management Ins and outs Task management
spinlocks

The mutex

Process of communication

A semaphore

Incident control

The message queue

 Memory allocation

Memory management

Memory assembly

The memory mapping

Rules of memory

Physical memory

 Total directory

Scheduling the story

Main memory slave

The source code comments

Source structure

Static site

 The clock task

Task scheduling

Task management

The scheduling queue

Scheduling mechanism

Thread concept

Concurrent parallel

The system calls

Task switching
The file system Hardware architecture
File concept

The file system

The index node

Mount the directory

Root file system

Character device

VFS

File handle

Pipeline file

 Compilation basis

Assembly and the cords

Working mode

register

Anomaly over

Assembly summary

Interrupt switch

Interrupt concept

Interrupt management

HongMeng station | into a little bit every day, the original is not easy, welcome to reprint, please indicate the source.