Microsoft’s 2019 Build developer conference was held in Redmond. Following the introduction of Bash Shell, native OpenSSH, WSL to Windows, and the availability of distributions like Ubuntu, SUSE Linux, and Fedora in the Microsoft Store, Microsoft has announced another big decision to bring the full Linux kernel to Windows 10.

According to Microsoft, Windows 10 will include a fully customized Linux kernel built in, providing better support for the latest WSL. The Linux kernel version 4.19**** is a long-term stable release. With the official release of WSL2, the Linux kernel for WSL2 will be fully open source. In recent years, Microsoft has hired a lot of Linux contributors, bringing many industry-leading Linux technologies to Microsoft, which is why WSL was created last year. Such a move will certainly attract more talents to join.

The Linux kernel, the most widely used open source kernel, is dominant in servers and smartphones, as well as hot technologies such as the Internet of Things, big data, cloud computing and artificial intelligence. For commercial companies, adopting the open source Linux kernel offers many benefits, including cost savings, the ability to take advantage of industry-leading technology, and the ability to customize and modify the kernel to their own needs. As for individuals, they can learn advanced design methods and programming techniques from the Linux kernel, and contribute code to the kernel to prove their technical strength.

However, when we are ready to learn the Linux kernel, we will find that the Linux kernel code is huge and complex, without the guidance of professional books, it is very difficult to read the code. How to learn the Linux kernel? This book is based on Linux 4.12 and the ARM64 hardware platform.




As an excellent open source system, Linux is widely used in the industry. Since 1994, when version 1.0 was released, the Linux kernel has been evolving, and the code is getting bigger and bigger, changing and changing, and getting to grips with it is no easy task.

Yu Huabing, the author of this book, has been engaged in Linux kernel development for more than ten years. He has rich practical experience and has used different versions and hardware platforms, from 2.x to 4.x, from MIPS, x86 to ARM. With the bright lights of the RESEARCH and development building, the students who just left school in those days are now blossoming and becoming an expert with profound accumulation in the field of Linux. I believe this book will make you fruitful.

Is it necessary for software engineers working on application development to learn the kernel?

Applications often use wrapped libraries that seem unrelated to the kernel, but are wrapped around the system calls provided by the kernel. Readers if the library function and kernel cooperate to achieve the function of the library functions provided, so the understanding of the software running process will be more profound, the skill level of the individual will be promoted to a new height, can design to develop high quality applications, in the process of the software runs to can quickly analyze the problems of positioning. In addition, the kernel represents the highest programming techniques in the software industry, and these programming techniques are also applicable to applications. Is it necessary to learn the kernel? The answer is yes.

Why learn Linux4.x?

In the early years, the operating systems of various vendors in the field of digital communication (including Cisco and Huawei) were developed based on traditional embedded operating systems (such as the typical VxWorks operating system). After 2010, the Linux kernel grew rapidly in the digital communication manufacturers and quickly became the kernel of the network operating system of the digital communication devices. Just as Linux-based Android has become the leader in smartphones, the Linux kernel is poised to dominate digital devices.

Some technical problems encountered in the development of network operating system based on Linux kernel have provided solutions in the evolution of Linux kernel. The ARM architecture of Linux 3.11 supports the giant page mechanism, which solves the performance problem of arm-based processes accessing large memory. Linux version 3.14 introduces ZRAM memory compression technology to save memory space, which is suitable for use on devices with small memory capacities. The Squash file system introduced in Linux 2.6.29 and overlay file system introduced in Linux 3.18 solve the problem of insufficient storage space on devices with small flash memory capacity. A squash file system can compress data, but it is a read-only file system, and devices need a writable file system. We found a solution in the Linux kernel that used an overlay file system to overlay a writable file system on top of a squash file system. These ready-to-use Linux kernel technologies are all covered in this book.

In recent years, Linux has developed rapidly. The earliest Linux kernel used by the company is version 2.6. From the release of version 3.0 in 2011 to the release of version 4.20 at the end of 2018, a total of 41 versions of Linux have been released, with rapid technological development. Many technologies are used and summarized by numerous engineers, but are still relatively fragmented, and Linux 4.x based books are few and far between.

What are the features of this book?

The code dissected in this book is based on Linux version 4.12, released in 2017, one of the most volatile releases in the history of the Linux kernel. Based on this version of the kernel code parsing and published as a book, is relatively new, not only does not lose universality, but also takes into account the introduction of many new technical points in 4.x version. At the same time, it is based on the ARM64 hardware platform, the combination of the two books, is still relatively few.

In addition, this book is not so much about the fundamentals of the operating system, but more about the actual code to understand how the various subsystems of the operating system are implemented in the Linux kernel. For readers familiar with the basics of the operating system, you can quickly cut into the understanding and implementation of specific code. Starting with kernel boot and initialization, through process management, memory management, interrupts/exceptions/system calls, kernel mutexes, and file systems, the book takes a fairly systematic and in-depth look at kernel code.

The Knowledge of the Linux kernel is too varied to cover in a single book, nor is it necessary. Therefore, in this book, seemingly inadvertently unfolded layer by layer knowledge points, are more basic and commonly used. The author with his practical experience as far as possible popular interpretation, and grasp the key points, so that readers in the actual development, debugging and maintenance work to learn.

“Action is the beginning of knowledge, and knowledge is the completion of action.” This is especially true of Learning Linux kernel technology. It takes a lot of engineering practice to really digest and understand the Linux kernel. I hope this book can be a good helper on your way forward!

How to read this book?

This book introduces the Linux kernel version 4.12. You are advised to download a copy of the Linux kernel code from the official website when reading this book. It is recommended to use Source Insight software to read the code.

The Linux kernel supports a variety of processor architectures, and the processor architecture-specific code is placed in the “ARCH” directory. ARM processors dominate mobile devices such as phones and tablets. ARM processor evolved from ARMv7 to support 64-bit ARMv8, ARM company redesigned the processor architecture, ARMv8 defines two execution states AArch64 and AArch32, AArch64 is the 64-bit architecture; AArch32 is a 32-bit architecture compatible with ARMv7. Because ARMv8 and ARMv7 are so different, the Linux kernel treats ARMv8 and ARMv7 as two different processor architectures, with the ARMv7 code in the “ARCH/ARM” directory and the ARMv8 code in the “ARCH/ARM64” directory. The AArch64 execution state of the ARMv8 architecture is often referred to as the ARM64 architecture. The ARM64 processor architecture is chosen for this book’s introduction to the Linux kernel.

To learn this book, you must have the basic knowledge of THE ARM64 processor. You are advised to download the following two documents from the ARM website.

(1) “ARM Cortex-A Series Programmer’s Guide for ARMV8-A” : this document is nearly 300 pages, suitable for getting started.

(2) “ARM Architecture Reference Manual ARMv8, for ARMV8-a Architecture Profile” : This document has more than 6000 pages, written in detail, suitable for Reference.

The key to learning the kernel is to understand the relationships between data structures and function calls. The complex relationship between data structures in the kernel, the deep level of function calls, the many branches in some functions, and the many techniques involved in a single function are all obstacles for beginners. The author advises readers to grasp the main clues as they learn, to understand the flow of execution, and not to pay too much attention to the details of functions at first. In order to facilitate learning, the author draws many diagrams to describe the relationship between data structures and the execution flow of functions. In addition, the author will introduce how to use each technology first, and I believe that it will be very helpful for readers to understand the technology.

The book is divided into six chapters.

Chapter 1 introduces the Boot process of the kernel. This book chooses the commonly used Boot program U-boot. Readers can download the U-boot code from the website of DENX Software Engineering Center in Germany and learn by comparison.

Chapter 2 introduces the technical principles of kernel management and scheduling processes.

Chapter 3 describes how the kernel manages virtual and physical memory.

Chapter 4 describes how the kernel handles exceptions and interrupts, and how system calls are implemented.

Chapter 5 describes the various mutually exclusive techniques implemented by the kernel to protect critical sections.

Chapter 6 covers the kernel’s virtual file system, which supports a variety of different file systems.




Deep Parsing of the Linux Kernel

Author: Yu Huabing

Editor’s Recommendation:

  • Linux 4.x kernel based on ARM64 architecture;

  • A large number of legends describe the relationship between data structures and the execution flow of functions;

  • Combined with code analysis, a comprehensive analysis of the Linux kernel in a generalized language.

Based on the 4.x Linux kernel, this book introduces the technical principles of several key subsystems of the Linux kernel. The main content of this book includes the kernel boot process, kernel management and scheduling process technology principle, kernel management virtual memory and physical memory technology principle, kernel handling exceptions and interrupts technology principle, as well as the implementation of system call and so on. In addition, the book details the various mutual exclusion techniques implemented by the kernel to protect critical sections, as well as the file system of the kernel. This book is rich in content, simple, through a large number of legends to describe the relationship between data structures and functions of the execution process, and combined with code analysis, guide readers to read and understand the kernel source code.

– END –