Small knowledge, big challenge! This paper is participating in theEssentials for programmers”Creative activities

Summary of a.

This paper will build the Docker image in the following ways, and obtain the fastest way to build Maven project in Docker by recording the construction time of each way:

  • Regular multi-phase build images
  • Build the image using Buildkit
  • Build the image using a dependency hierarchy
  • Use volume mounts during Buildkit builds
  • Build the image using the Maven daemon

Between each run, we change the source code by adding a blank line; Between each section, we removed all built images, including intermediate images as a result of a multi-phase build, in order to avoid reusing previously built images in order to get a more accurate build time for each approach. Let’s test with a simple Spring Boot project.

Ii. Conventional multi-stage construction mirror

Here’s the relevant Dockerfile:

FROM openjdk:11-slim-buster as build                         

COPY .mvn .mvn                                               
COPY mvnw .                                                  
COPY pom.xml .                                               
COPY src src                                                 

RUN ./mvnw -B package                                        

FROM openjdk:11-jre-slim-buster                              

COPY- the from = build target/fast - maven - builds - 1.0. The jar.

EXPOSE 8080

ENTRYPOINT ["java"."-jar"."Fast - maven - builds - 1.0. The jar"]     
Copy the code

Let’s do the build:

Time DOCKER_BUILDKIT=0 docker build-t fast-maven:1.0.Copy the code

Forget environment variables for a moment, which I’ll explain in the next section

Here are the results of five runs:

* 0.36s user 0.53s system 0% cpu 1:53.06 total
* 0.36s user 0.56s system 0% cpu 1:52.50 total
* 0.35s user 0.55s system 0% cpu 1:56.92 total
* 0.36s user 0.56s system 0% cpu 2:04.55 total
* 0.38s user 0.61s system 0% cpu 2:04.68 total
Copy the code

Build images with Buildkit

The previous command line uses the DOCKER_BUILDKIT environment variable, which is a way of telling Docker to use the old engine. If you haven’t updated Docker in a while, that’s the engine you’re using. Today, BuildKit has replaced it as the new default.

BuildKit brings a number of performance improvements:

  • Automatic garbage collection
  • Concurrent dependency resolution
  • Efficient instruction caching
  • Build the cache import/export
  • And so on.

Let’s re-execute the previous command on the new engine:

Time Docker build-t fast-maven:1.1.Copy the code

Here’s an excerpt of the console log from the first run:

. => => transferring context: 4.35kB => [Build 2/6] COPY.mvn. MVN => [Build 3/6] COPY MVNW. => [Build 4/6] COPY pop.xml. => [Build 5/6] COPY SRC src => [build 6/6] RUN ./mvnw -B package ... 0.68s user 1.04s System 1% CPU 2 06.33 TotalCopy the code

The following execution of the same command has slightly different output:

. => => transferring context: 1.82kB => CACHED [build 2/6] copy.mvn. MVN => CACHED [Build 3/6] COPY mvnw. => CACHED [build 4/6] COPY pom.xml. => [build 5/6] COPY src src => [build 6/6] RUN ./mvnw -B package ...Copy the code

Keep in mind that we changed the source code between runs. We won’t change the files, i.e. MVN, MVNW and POM.xml, cached by BuildKit. But these resources are small, so caching does not significantly improve build times.

* 0.69s user 1.01s system 1% cpu 2:05.08 total
* 0.65s user 0.95s system 1% cpu 1:58.51 total
* 0.68s user 0.99s system 1% cpu 1:59.31 total
* 0.64s user 0.95s system 1% cpu 1:59.82 total
Copy the code

A quick look at the logs reveals that the biggest bottleneck in the build is the download of all dependencies, including plug-ins. This happens every time we change the source code, which is why BuildKit doesn’t improve performance.

Build an image using a dependency hierarchy

We should focus our efforts on dependencies. To do this, we can leverage layers and break the build into two steps:

  • The first step is to download the dependency
  • In the second, we do proper packaging

Each step creates a layer, the second depending on the first.

By layering, if we change the source code at the second layer, the first layer is not affected and can be reused. We do not need to download the dependency again. The new Dockerfile looks like:

FROM openjdk:11-slim-buster as build COPY .mvn .mvn COPY mvnw . COPY pom.xml . RUN ./mvnw -B dependency:go-offline COPY SRC SRC RUN./ mvnw-b package FROM OpenJDK :11-jre-slim-buster COPY -- FROM =build target/fast-maven-builds-1.2.jar. EXPOSE 8080 ENTRYPOINT [" Java ", "-jar", "Fast-Maven-Builds-1.2.jar "]Copy the code

Note: Go-Offline does not download everything. If you try to use the -o option (for offline), the command will not run successfully. This is a well-known old mistake. In all cases, it was “good enough.”

Let’s run the build:

Time Docker build-t fast-maven:1.2.Copy the code

The first run takes more time than a regular build:

0.84s user 1.21s system 1% cpu 2:35.47 total
Copy the code

However, subsequent builds are much faster. Changing the source code affects only the second layer and does not trigger the download of (most) dependencies:

* 0.23s user 0.36s system 5% cpu 9.913 total
* 0.21s user 0.33s system 5% cpu 9.923 total
* 0.22s user 0.38s system 6% cpu 9.990 total
* 0.21s user 0.34s system 5% cpu 9.814 total
* 0.22s user 0.37s system 5% cpu 10.454 total
Copy the code

Use volume mounts during Buildkit builds

Layered building greatly reduces build time, but there is a problem that changing individual dependencies invalidates the mirrored dependency layer, so we need to download all dependencies again.

Fortunately, BuildKit introduces volume mounts during build time, not just during run time. There are several types of mounts available, but one we are interested in is cache mounts. This is an experimental feature, so you need to explicitly opt in:

Dockerfile looks like:

# syntax=docker/dockerfile:experimental                      
FROM openjdk:11-slim-buster as build

COPY .mvn .mvn
COPY mvnw .
COPY pom.xml .
COPY src src

# Build with cache
RUN --mount=type=cache,target=/root/.m2,rw ./mvnw -B package 

FROM openjdk:11-jre-slim-buster

COPY- the from = build target/fast - maven - builds - 1.3. The jar.

EXPOSE 8080

ENTRYPOINT ["java"."-jar"."Fast - maven - builds - 1.3. The jar"]
Copy the code

Where # syntax = docker/dockerfile: experimental function is used to open experiment.

Run the following command to build the image:

Time Docker build-t fast-maven:1.3.Copy the code

Build times are higher than regular builds, but still lower than hierarchical builds:

0.71s user 1.01s system 1% cpu 1:50.50 total
Copy the code

The following constructs are equivalent to layers:

* 0.22s user 0.33s system 5% cpu 9.677 total
* 0.30s user 0.36s system 6% cpu 10.603 total
* 0.24s user 0.37s system 5% cpu 10.461 total
* 0.24s user 0.39s system 6% cpu 10.178 total
* 0.24s user 0.35s system 5% cpu 10.283 total
Copy the code

Build the image using Maven daemon

The contents of the Maven daemon Dockerfile are as follows:

FROM openjdk:11-slim-buster as build
Download the latest version of the Maven daemon
ADDhttps://github.com/mvndaemon/mvnd/releases/download/0.6.0/mvnd-0.6.0-linux-amd64.zip.
Update package index
RUN apt-get update \     
# installed unzip
 && apt-get install unzip \     
Create a private folder
 && mkdir /opt/mvnd \       
Extract the MVND we downloaded earlier
 && unzip mvnd-0.6.0-linux-amd64.zip \ 
Move the extracted archive content to the folder you created earlier
 && mv mvnd-0.6.0-linux-amd64/* /opt/mvnd                    

COPY .mvn .mvn
COPY mvnw .
COPY pom.xml .
COPY src src
# Use MVND instead of Maven wrapper
RUN /opt/mvnd/bin/mvnd -B package                            

FROM openjdk:11-jre-slim-buster

COPY- the from = build target/fast - maven - builds - 1.4. The jar.

EXPOSE 8080

ENTRYPOINT ["java"."-jar"."Fast - maven - builds - 1.4. The jar"]
Copy the code

Build the image using the following command:

Time Docker build-t fast-maven:1.4.Copy the code

The following output is displayed:

* 0.70s user 1.01s system 1% cpu 1:51.96 total
* 0.72s user 0.98s system 1% cpu 1:47.93 total
* 0.66s user 0.93s system 1% cpu 1:46.07 total
* 0.76s user 1.04s system 1% cpu 1:50.35 total
* 0.80s user 1.18s system 1% cpu 2:01.45 total
Copy the code

There is no significant improvement over regular build images.

Conclusion seven.

Here is a summary of all execution times:

The baseline Build Kit The layer Volume mount MVND
#1 (S) 113.06 125.08 155.47 110.5 111.96
#2 (S) 112.5 118.51 9.91 9.68 107.93
#3 (S) 116.92 119.31 9.92 10.6 106.07
#4 (S) 124.55 119.82 9.99 10.46 110.35
#5 (S) 124.68 9.81 10.18 121.45
#6 (S) 10.45 10.28
#7 (S) 44.71
Average (seconds) 118.34 120.68 9.91 10.24 111.55
deviation 28.55 6.67 0.01 0.10 111.47
Baseline gain (S) 0 2.34 108.43 108.10 6.79
% for 0.00% – 1.98% 91.63% 91.35% 5.74%

The performance of speeding up Maven builds in Docker is quite different from regular builds, with the limiting factor being the download speed of dependencies and the need to use layers to cache dependencies.

For BuildKit, it is recommended to use the new cache mount capability to avoid downloading all dependencies when the layer fails.

Refer to the article

Blog. Frankel. Ch/faster – – mave…