Moment For Technology

Docker containers are parsed from the shallow to the deep

Posted on Sept. 27, 2022, 11:31 a.m. by Mrs. Gemma Holt
Category: The back-end Tag: docker

Used to introduce

Fast installation

Refer to the official documentation: docs.docker.com/engine/inst...

The following is a brief introduction to the MAC installation.

MAC installation

Click install

Eventually launch Docker from the app

Common commands

In general, we can use the Docker Desktop to configure. But I am more used to using the command line, the base or can learn the next. This section focuses on a few common commands. Here, I'll use a small example to introduce a few common commands.

Scenario: I need to build a mysql-dependent application image and start a container runtime with that image. After the running starts properly, stop the service and push the image to the remote repository.

Start mysql container

#If you need to specify the CPU architecture, add -- Platform Linux/AMd64 on the command lineDocker pull mysql: 5.7.19#To start the mysql container, add --platform Linux /amd64 on the command lineDocker run-itd --name mysql-test -p 3306:3306 -e MYSQL_ROOT_PASSWORD=123456 mysql:5.7.19
## Mariadb: No arm image for MAC M1
docker pull mariadb
#Start mariadb and set root password to 123456
docker run -p 3306:3306  --name some-mariadb -e MARIADB_ROOT_PASSWORD=123456 -d mariadb
#Check whether the database is normal
docker ps |grep mariadb
Copy the code

Building an Application Image

We can add a Dockerfile to our code

The directory structure is as follows:

package main

import (
	"database/sql"
	"fmt"
	_ "github.com/go-sql-driver/mysql"
)

var db *sql.DB // Declare a global db variable

// Initialize the MySQL function
func initMySQL(a) (err error) {
  // Connect to the local database, connect to the database test_db
	dsn := "Root: 123456 @ TCP (127.0.0.1:3306)/test_db"
	db, err = sql.Open("mysql", dsn)
	iferr ! =nil {
		return
	}
	err = db.Ping()
	iferr ! =nil {
		return
	}
	return
}
func main(a) {
	// Initialize MySQL
	err := initMySQL()
	iferr ! =nil {
		panic(err)
	}
	defer db.Close()
	fmt.Println("mysql connection is successful")
	select{}}Copy the code

Go build -o dockerTest and edit the Dockerfile as follows:

FROM centos
COPY dockerTest /opt/
CMD ["/opt/dockerTest"]
Copy the code
## Build image
docker build -t docker-test .

## View image
docker images

#Mysql  connect to mysql  mysql  mysql  mysql  mysql  Mysql  Mysql  Mysql  Mysql  mysql
docker run -it --rm --network host docker-test

## You can push the image to the remote warehouse after the image is built. First, re-tag the image and then push itDocker tag docker - test XXX. Domain. Cn/test/docker - test: 1.0 docker push docker - test XXX. Domain. Cn/test/docker - test: 1.0Copy the code

This completes one of the above scenarios.

Command to introduce

The above operation process basically covers the common Docker command line, here is attached to a common command line classification

The command The sample instructions
Docker Environment information Info, version
Container lifecycle management Create, exec, kill, pause, restart, rm, run, start, stop, unpause
Mirror warehouse command Login, logout, pull, push, and search
Image management Build, images, import, load, RMI, save, Tag, commit Build, Tag, images, and RMI are commonly used
Container oM operations Attach, export, inspect, port, ps, rename, STATS, top, wait, CP, DIff, UPDATE Inspect is a common command for viewing container information
Container Resource Management Volume, the network
System log information Events, history, and logs

The following is a transition diagram

If you need more detailed method of use, available at www.runoob.com/docker/dock...

Common scenario

Elasticsearch tools

hub.docker.com/_/kibana

Docker run-d --name Kibana --network host-p 5601:5601 Kibana :6.8.17Copy the code

Redis installation

hub.docker.com/_/redis

docker run -p 6379:6379 --name some-redis -d redis redis-server --appendonly yes
Copy the code

The principle is introduced

This is all about the use layer. If you're a beginner, you might have a lot of doubts. When I was using containers, one of the things that puzzled me the most was, how does Docker build a program once and run it everywhere? So let's focus on that.

When we look at this problem, the first thing we think about is the mirroring mechanism of the container. Image is responsible for a package of the program, no matter what system, as long as docker is deployed, pull it down from the image warehouse, we can use Docker to run the program.

Mirror image principle

Image definition

Docker image is a read-only Docker container template, containing the file system structure and its contents required to start a Docker container, so it is the basis for starting a Docker container. The file content of the Docker image and the configuration of some running Docker container constitute the static file system running environment of the Docker container ----rootfs. It can be understood that Docker image is the static perspective of Docker container, and Docker container is the running state of Docker image

rootfs

Rootfs is the file system visible to the internal process of the Docker container when it is started, that is, the root directory of the Docker container. This typically includes typical directory systems such as /dev/proc, /bin, /etc, /lib, /usr, / TMP, and the configuration files and tools needed to run the Docker container. An image is simply a configuration that mirrors the contents of the file to the machine

Image features

layered

Docker images can be built in a hierarchical way, with each image consisting of a series of "mirror layers". In this way, the image layer can be shared between different mirrors.

When writing copy

Docker image uses copy-on-write policy to share images among multiple containers. Each container does not need to copy an image file separately when it starts. Instead, all mirror layers are mounted to a mount point in a read-only way, and a container layer of read and write layer is overwritten on top. When a write occurs, the contents of the file are written to the read/write layer and the older version of the file in the read/write layer is hidden. Replication on write combined with the hierarchical mechanism reduces the disk space consumed by mirroring and the container startup time.

Content addressing

Each image layer of a Docker image will carry out a content calculation and verification to generate a content hash value, and the hash value will be used as the unique identifier of the image layer. This mechanism provides security for mirroring.

Joint mount

Federated mount allows multiple filesystems to be mounted simultaneously at a single mount point, consolidating the contents of the mount points so that the final visible filesystems contain the consolidated layers of files and directories.

Docker image storage organization

Now that we've mentioned some of the features of mirrors, how are they actually organized? We can see a complete, running container with all the file system structure.

Image related Concepts

registry

Registry is used to hold Docker images, which also includes the image hierarchy and metadata about the image. It's easy to think of Registry as an entity like a Git repository.

repository

Repository is a mirror group consisting of all iterations of a Docker image with a specific function. Registry is a collection of Repositories and Repository is a collection of mirrors.

manifest

Manifest mainly exists in Registry as the metadata file of Docker image, and as the description file of image structure and basic information in pull, push, save and load.

Image and layer

The image concept inside Docker is used to store a group of image-related metadata information, mainly including image architecture (such as AMD64), image default configuration information, image container configuration information, and rootFS containing all image layer information. Docker uses Diff_id in RootFS to calculate the index (ChainID) of content addressing to obtain the relevant information of layer, and then obtain the file content of each mirror layer.

Layer is an intermediate concept that Docker uses to manage images. The Layer in Docker image management mainly stores diff_id, size, cache-ID, parent and other contents of the image layer. The actual file content is managed by the storage driver and can be locally indexed through cache-ID.

Dockerfile

Dockerfile allows the user to define a Docker image using the basic DSL syntax, with each instruction describing the steps to build the image. The definition file we need to use when building our own Docker image using the docker build command.

Docker image build

Based on some of the previous concepts, we can try to think about what problem is it trying to solve? Why make it so complicated? Wouldn't it be nice if I just tar it?

Docker image packaging actually provides a good delivery standard. In general, we can only build the modules dependent on the involved code together, while the image can package the operating system level dependencies together. If we want to implement this, we will face a series of problems, and Docker will solve them one by one.

  1. How to orchestrate this build, docker uses Dockerfile
  2. How to build, Docker build
  3. How to manage image, docker pull, docker push

And this standard of mirror building, how does it work?

Build the implementation process

Let's use the following command as an example.

Docker build-t XXX :1.0.Copy the code

The build process focuses on packaging the context, which is the current directory. Docker service architecture is C/S architecture, that is, the pattern of client and server.

The client will read the current directory, package it and send it to the Docker server. After receiving the request, the server will do the following actions:

  1. Create a temporary directory and unzip the file system specified by context into that directory.
  2. Read and parse the Dockerfile.
  3. According to the parsed Dockerfile, all the instructions in it are traversed and distributed to different modules for execution.
  4. Parser creates a temporary container for each of the above instructions, executes the current instruction in the temporary container, and then uses this container to generate a mirror layer with commit.
  5. The combination of all the corresponding layers in the Dockerfile directive is the result of this build.

Docker image metadata management

The mirror entity here, the ultimate embodiment, is also presented in a file system manner. Storage management Different storage drivers have different mirror implementations. We will analyze overlay2, which is commonly used at present.

The repository metadata

A repository mentioned above is an image library consisting of all iterations of a Docker image with a certain function. The repository in local persistence files stored in the/var/lib/docker/images/overlay2 repositoryies. Json file.

This file stores the names of all Repositories, the names and tags of all version images in each Repository, and the corresponding image IDS. ReferenceStore is used to resolve the different formats of the Repository name and manage the mapping between the Repository and the image ID.

Image metadata

ImageStore management mirror ID and the mapping relationship between image metadata and metadata persistence operation, persistent files generally located in the/var/lib/docker/image/overlay2 / imagedb/content/sha256 / [image_id]

Layer metadata

Mirroring is managed through a graph structure. Each mirroring layer has metadata, which records the building information of this layer and the ID of the parent mirror layer, while the top mirroring layer records some more information as the metadata of the whole mirror. Graph maintains a tree-like structure of mirror layers based on the mirror ID (that is, the top mirror layer ID) and the parent mirror ID of each mirror layer record.

The structure of layer is a little more complicated, so we can understand the following points first:

  1. RoLayer and mountedLayer. RoLayer is used to describe the immutable mirror layer and mountedLayer is used to describe the read-write container layer.
  2. RoLayer involves two ID, inside the file stored in the/var/lib/docker/image/overlay2 / layerdb sha56 / [chainID]
    1. DiffID is obtained by using SHA256 algorithm based on the content of the image layer packet
    2. ChainID is the index based on the content store, which is calculated based on diffID between the current layer and all ancestor mirror layers.
  3. The mountedLayer stores the ID of a container that can be read or written, the ID of the container init layer in the GraphDriver --initID, The ID of the read-write layer in graphDriver --mountID and the chainID--parent of the parent mirror layer of the container layer. Persistent files in the/var/lib/docker/image/overlay2 / layerdb/mounts / [container_id]

conclusion

  1. The installation of Docker on MAC computer is briefly introduced
  2. Based on examples, the use of Docker is introduced. Common commands include build, run, pull, push and so on
  3. The principle of Docker image mainly uses layered mechanism, writing-on-copy, content addressing, and joint mounting technologies to realize the construction of image.
  4. The entire build process is mainly done on the server side, and the final product of the build is placed under /var/lib/docker/image.

conclusion

So the mirror principle basically introduces, what does the mirror have to do with the container, how does the container run the program?

If you think the content is good, want to study IT technology together, you can close my public number: gungunXI. Or add my wechat id: lcomedy2021

The resources

Official image: github.com/docker-libr...

The docker common administrative commands: segmentfault.com/a/119000000...

Parsing docker images: linux.cn/article-607...

Search
About
mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.