Use container mode to compile Drone CI without functional limitations

Because the default version of Drone contains a limit on the number of build times, if we use Drone frequently in daily life, we will soon encounter the problem of needing to “re-initialize” the application to continue to use it. But in fact, as individual users, we are actually not affected by this limit.

Therefore, this article will share how to use containers to build Drone CI without restrictions.

Writing in the front

Why did this article appear? Because “Childe”, a former classmate in the group, once mentioned that “Drone default mirror is the enterprise version, which has a limit of 5000 times to build, and needs to be recompiled”. Considering the long-term stability of the software to use, to read the official document, see that were true, the document is so described: “there are two versions of the Drone, respectively is needed to build a community of open source version, and official offer enterprise version”, but the official did not more for compiling construction related documents or instructions.

About the Drone CI

For individual developers or teams, we are most concerned about whether the code is secure, how the software is licensed, and how much it costs. The official documentation states:

The software is all open source, and the official default is to provide everyone with the enterprise version of the mirror trial, supporting 5000 build calls.
If you want to use an open source version, you need to build it yourself.
The Enterprise edition is free for personal use.
It’s free if your team, company (including nonprofit) has less than $1 million in annual revenue, or raises less than $5 million.
If your company pays, the minimum threshold is $299 per month.

In the official enterprise service page, we can see the function differences of different versions, mainly depends on whether they support: running multiple runners in distributed mode; Use K8S Runner; Support organization key function; It can be used with Vault. Supports scheduled tasks. Postgres, mysql, and S3 are used for data storage. Supports automatic capacity expansion and Extended functions.

If you have the above requirements, you can build an “enterprise edition” or vice versa. However, combined with the above limitations, it is easier and more comprehensive for the average individual and team to build the enterprise edition directly.

So, let’s take a look at how to take a similar “official way” to build Drone’s container image.

Collect Drone official construction information

Scrolling through the documentation, you can see that there are only two (or one) simple commands for building your own:

# Build an open source version
$ go build -tags "oss nolimit" github.com/drone/drone/cmd/drone-server

# Build the Enterprise edition
$ go build -tags "nolimit" github.com/drone/drone/cmd/drone-server
Copy the code

In order to build an official mirror, a complete “build routine” needs to be combed out from the official repository. Here, based on the V1.10.1 code, the construction method is sorted out.

From the BUILDING and BUILDING_OSS files at the root of the repository, you can see that the installation and build processes for the two distributions are documented:

Clone the repository 2. Install Go 1.11 or later with go modules enabled 3$GOPATH/bin

    go install -tags "oss nolimit" github.com/drone/drone/cmd/drone-server

4. Start the server at localhost:8080

    export DRONE_GITHUB_CLIENT_ID=...
    export DRONE_GITHUB_CLIENT_SECRET=...
    drone-server
Copy the code

Continue scrolling through the project’s.Drone. yml CI file to see how officials build and distribute software through CI:

.
- name: build
  image: Golang: 1.14.4
  commands:
  - sh scripts/build.sh
  environment:
    GOARCH: amd64
    GOOS: linux

- name: publish
  image: plugins/docker:18
  settings:
    auto_tag: true
    auto_tag_suffix: linux-amd64
    dockerfile: docker/Dockerfile.server.linux.amd64
    repo: drone/drone
    username:
      from_secret: docker_username
    password:
      from_secret: docker_password
  when:
    event:
    - push
    - tag
.
Copy the code

Take a look at the “build script” mentioned in the CI document, which reads as follows:

#! /bin/sh

echo "building docker images for ${GOOS}/${GOARCH} ..."

REPO="github.com/drone/drone"

# compile the server using the cgo
go build -ldflags "-extldflags \"-static\"" -o release/linux/${GOARCH}/drone-server ${REPO}/cmd/drone-server

# compile the runners with gcc disabled
export CGO_ENABLED=0
go build -o release/linux/${GOARCH}/drone-agent      ${REPO}/cmd/drone-agent
go build -o release/linux/${GOARCH}/drone-controller ${REPO}/cmd/drone-controller
Copy the code

Continue to check the container Dockerfile docker/Dockerfile. Server. Linux. Amd64, can see container structure:

# docker build --rm -f docker/Dockerfile -t drone/drone .FROM Alpine :3.11 as Alpine RUN apk add-u --no-cache ca-certificates FROM Alpine :3.11 EXPOSE 80 443 VOLUME /data RUN [!  -e /etc/nsswitch.conf ] &&echo 'hosts: files dns' > /etc/nsswitch.conf

ENV GODEBUG netdns=go
ENV XDG_CACHE_HOME /data
ENV DRONE_DATABASE_DRIVER sqlite3
ENV DRONE_DATABASE_DATASOURCE /data/database.sqlite
ENV DRONE_RUNNER_OS=linux
ENV DRONE_RUNNER_ARCH=amd64
ENV DRONE_SERVER_PORT=:80
ENV DRONE_SERVER_HOST=localhost
ENV DRONE_DATADOG_ENABLED=true
ENV DRONE_DATADOG_ENDPOINT=https://stats.drone.ci/api/v1/series

COPY --from=alpine /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

ADD release/linux/amd64/drone-server /bin/
ENTRYPOINT ["/bin/drone-server"]
Copy the code

Now that we have the clues, we can start writing the basic container image.

Writing build scripts

In the previous section, we can see that the core software drone-server used in the container files is copied from “local” to the image. It is suspected that this strategy is adopted for higher compilation efficiency and shorter compilation time of binaries used by multiple platforms.

For our purposes, we only need to use software suitable for a particular CPU architecture and system, so we can consider creating the compilation environment directly in containers. In addition to better preserving the compilation environment, it can also make the compilation machine system environment more “pure and clean” :

FROM golang:1.16.0- Alpine3.13 AS Builder ENV DRONE_VERSION 1.10.1 ENV CGO_CFLAGS="-g -O2 -Wno-return-local-addr"

RUN apk add build-base && go env -w GO111MODULE=on && \
    mkdir /src && cd /src && \
    apk add curl && curl -L https://github.com/drone/drone/archive/refs/tags/v${DRONE_VERSION}.tar.gz -o v${DRONE_VERSION}.tar.gz && \
    tar zxvf v${DRONE_VERSION}.tar.gz && rm v${DRONE_VERSION}.tar.gz && \
    cd /src/drone-${DRONE_VERSION} && \
    go mod download && \
    go build -ldflags "-extldflags \"-static\"" -tags="nolimit" github.com/drone/drone/cmd/drone-server
Copy the code

In order to speed up the build, we can adjust the Dockerfile appropriately and add some domestic software sources:

FROM Golang :1.16.0- Alpine3.13 AS Builder RUN sed -I's/https:\/\/dl-cdn.alpinelinux.org/http:\/\/mirrors.tuna.tsinghua.edu.cn/' /etc/apk/repositories && \
    echo "Asia/Shanghai"> /etc/timezone RUN apk add build-base && \ go env -w GO111MODULE=on && \ go env -w GOPROXY=https://mirrors.aliyun.com/goproxy/, direct ENV DRONE_VERSION 1.10.1 WORKDIR/SRC# Build with online code
RUN apk add curl && curl -L https://github.com/drone/drone/archive/refs/tags/v${DRONE_VERSION}.tar.gz -o v${DRONE_VERSION}.tar.gz && \
    tar zxvf v${DRONE_VERSION}.tar.gz && rm v${DRONE_VERSION}.tar.gz
# OR with offline tarball
# ADD drone - 1.10.1. Tar. Gz/SRC /

WORKDIR /src/drone-${DRONE_VERSION}

RUN go mod download

ENV CGO_CFLAGS="-g -O2 -Wno-return-local-addr"

RUN go build -ldflags "-extldflags \"-static\"" -tags="nolimit" github.com/drone/drone/cmd/drone-server
Copy the code

Save the above content as Dockerfile, and then execute the Docker build-t drone:1.10.1. Wait for a moment, the “fully functional” drone is built in the image, but the image size is very large, fully 1.28GB. So we’ll go ahead and write a multi-phase build image to reduce the container size.

Multi-phase mirror build

Adding a few more things below the container declaration file above, we can make some adjustments to the build script in conjunction with the official build script found above:

FROM Golang :1.16.0- Alpine3.13 AS Builder RUN sed -I's/https:\/\/dl-cdn.alpinelinux.org/http:\/\/mirrors.tuna.tsinghua.edu.cn/' /etc/apk/repositories && \
    echo "Asia/Shanghai"> /etc/timezone RUN apk add build-base && \ go env -w GO111MODULE=on && \ go env -w GOPROXY=https://mirrors.aliyun.com/goproxy/, direct ENV DRONE_VERSION 1.10.1 WORKDIR/SRC# Build with online code
RUN apk add curl && curl -L https://github.com/drone/drone/archive/refs/tags/v${DRONE_VERSION}.tar.gz -o v${DRONE_VERSION}.tar.gz && \
    tar zxvf v${DRONE_VERSION}.tar.gz && rm v${DRONE_VERSION}.tar.gz
# OR with offline tarball
# ADD drone - 1.10.1. Tar. Gz/SRC /

WORKDIR /src/drone-${DRONE_VERSION}

RUN go mod download

ENV CGO_CFLAGS="-g -O2 -Wno-return-local-addr"

RUN go build -ldflags "-extldflags \"-static\"" -tags="nolimit"Github.com/drone/drone/cmd/drone-server FROM alpine: 3.13 AS Certs RUN sed - I's/https:\/\/dl-cdn.alpinelinux.org/http:\/\/mirrors.tuna.tsinghua.edu.cn/' /etc/apk/repositories && \
    echo "Asia/Shanghai"> /etc/timezone RUN apk add -u --no-cache ca-certificates FROM alpine:3.13 EXPOSE 80 443 VOLUME /data RUN [! -e /etc/nsswitch.conf ] &&echo 'hosts: files dns' > /etc/nsswitch.conf

ENV GODEBUG netdns=go
ENV XDG_CACHE_HOME /data
ENV DRONE_DATABASE_DRIVER sqlite3
ENV DRONE_DATABASE_DATASOURCE /data/database.sqlite
ENV DRONE_RUNNER_OS=linux
ENV DRONE_RUNNER_ARCH=amd64
ENV DRONE_SERVER_PORT=:80
ENV DRONE_SERVER_HOST=localhost
ENV DRONE_DATADOG_ENABLED=trueENV DRONE_DATADOG_ENDPOINT=https://stats.drone.ci/api/v1/series COPY --from=Certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/copy --from=Builder/SRC /drone-1.10.1/drone-server /bin/drone-server ENTRYPOINT ["/bin/drone-server"]
Copy the code

Docker build-t drone:1.10.1. Docker build-t Drone :1.10.1. Docker Build-t Drone :1.10.1.

other

Earlier this year, I wrote an article about Drone: “Lightweight Warehouse and CI use scheme in container Mode: Gitea + Drone foundation article, a few days ago in the “site optimization log (2021.04.12)”, also mentioned that I was trying to use Gitea + Drone to replace the GitLab before personal use, so if you have similar lightweight operation requirements, you can read the previous article, It might save us some time in the process.

Of course, if you are interested in GitLab Runner build, you can read a content two years ago: source code Build GitLab Runner, also written using Golang, but by comparison, much more complex than Drone.

The last

I hope this article can help you to use Drone.

–EOF

We have a little group of people who like to do things.

In the case of no advertisement, we will talk about software and hardware, HomeLab and programming problems together, and also share some information of technical salon irregularly in the group.

Like to toss small partners welcome to scan code to add friends. (Please indicate the source and purpose, and note the real name, otherwise it will not pass the audit)

All this stuff about getting into groups

This article is published under a SIGNATURE 4.0 International (CC BY 4.0) license. Signature 4.0 International (CC BY 4.0)

Author: Su Yang

Creation time: April 17, 2021 statistical word count: 7325 words reading time: 15 minutes to read this article links: soulteary.com/2021/04/17/…

Use container mode to compile Drone CI without functional limitations

Writing in the front

About the Drone CI

Collect Drone official construction information

Writing build scripts

Multi-phase mirror build

other

The last

Related Posts

Tinydb source code read

JVM Series 5: JVM tuning Examples

One Linux command per day (5) : rm command