This article is published under a SIGNATURE 4.0 International (CC BY 4.0) license. Signature 4.0 International (CC BY 4.0)

Author: Su Yang

Creation time: August 5, 2019 statistical word count: 7023 words reading time: 15 minutes of reading this article links: soulteary.com/2019/08/05/…

GitLab security GitLab security GitLab security GitLab security GitLab security

GitLab, which is built on the public network, frequently faces security challenges. However, with only one or two simple actions, maintenance costs can be greatly reduced and unlicensed content can be avoided from being exposed by search engine crawlers everywhere.

In this article, we will discuss the security details of the GitLab code repository set up by the public network.

Writing in the front

The common attack surfaces for setting up GitLab on the public network are as follows:

  • Run the host system part
  • Run the network part of the host
  • Application Web vulnerability
  • Applying SSH Vulnerability

SLB + VPC can be used for network isolation to reduce attack risks. In addition to following up system security patches and upgrading application versions as soon as possible, there are better solutions. After all, programs storing data have unknown risks every time they are upgraded:

  • Web vulnerabilities can be addressed by adding a layerBasic AuthTo solve it.
  • The risk of SSH attacks can be addressed by adding a simple log monitor: refer to the GitLab SSH Port section in the previous article.

But adding a layer of Basic Auth actually causes some problems with GitLab.

Add request validation for GitLab

The GitLab application itself does not support Basic Auth, and requires a Web front-end software such as Nginx and Traefik to do this.

I chose to use Traefik because it is easier to configure, as described in the “Adding Network Request Authentication” section of the previous article.

It is enough to add a sentence to the configuration declaration, such as:

- "traefik.gitlab.frontend.auth.basic=soulteary:$apr1$E86fARwM$tXmggGAtCEDKqsBCSvDA3/
Copy the code

After this is done, all HTTP requests are validated for legitimate access. Other ports and protocols, such as the SSH service on port 22, are not affected.

What exactly is Basic Auth

When you access the page, a dialog box similar to the following is displayed asking the user to log in. Otherwise, 401 Unauthorized users will be prompted.

The same is true when a crawler/security checker requests a page without submitting a user name and password.

curl -I https://gitlab.domain/
HTTP/2 401
content-type: text/plain
vary: Accept-Encoding
www-authenticate: Basic realm="traefik"
content-length: 17
date: Sat, 03 Aug 2019 18:22:17 GMT
Copy the code

If you enter the correct username and password, you will see an additional request parameter, authorization, appear in your request parameters.

The value usually consists of two parts: the first part indicates the encryption method (the name of the encryption protocol) and the second part indicates your identity (see RFC 7617 for more information).

Modern browsers are smart enough to record the identity information after you enter it correctly the first time and carry it with them in each subsequent request. If you use an application or tool, you need to manually add the authorization information to each HTTP request header.

However, Basic Auth blocked the casual access of outsiders as well as the GitLab CI Runner.

So what’s going on here? Let’s move on.

Rescue a blocked CI Runner

To explain why CI Runner was blocked by Basic Auth, we need to look at another protocol specification, RFC1738, which defines HTTP:

//<user>:<password>@<host>:<port>/<url-path>
Copy the code

When we use clients (browsers, curl, etc.) to request addresses in this format, some clients convert the User: Password part into standard HTTP Authorization request headers.

Default CI Runner behavior

If no changes are made, an error is reported when CI is executed directly, and the log output is similar to the following:

Running with gitlab-Runner 12.0.2 (d0b76032) on Ci-Runner sGNUJnq6 Using Shell Executor Running with Gitlab-Runner 12.0.2 (d0b76032) on Ci-Runner sGNUJnq6 Using Shell Executor... Running on ci-runner... Fetching changes with git depthset to 50...
Initialized empty Git repository in /data/runner/builds/sGNUJnq6/0/config/project/.git/
Created fresh repository.
remote: 401 Unauthorized
fatal: Authentication failed for 'https://gitlab-ci-token:[MASKED]@gitlab.domain/repo.git/'
ERROR: Job failed: exit status 1
Copy the code

Combined with the previous content of the article, we know that the Authorization request header is missing, so we try to add this request header to the request, and build a Proxy between CI and GitLab, so that CI can automatically complete “authentication” when requesting GitLab data.

Request to automatically add authentication information

If you use Nginx to build a proxy to support GET/POST requests, the core configuration is as follows:

Location / {forward-http-post-requests-via-rewrite proxy_pass https://192.168.0.123; Proxy_redirect https://192.168.0.123/ /; proxy_read_timeout 10s; proxy_set_header Host'gitlab.domain';
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Authorization 'Basic YmFhaABCD';
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto 'https';
    proxy_http_version 1.1;
}
Copy the code

Delete from GitLab: delete from GitLab: delete from GitLab

curl -I https://gitlab.domain/ HTTP/2 302 cache-control: no-cache content-type: text/html; charset=utf-8 date: Sat, 03 Aug 2019 18:45:39 GMT location: https://gitlab.domain/users/sign_in referrer-policy: strict-origin-when-cross-origin referrer-policy: strict-origin-when-cross-origin server: nginx strict-transport-security: max-age=315360000 vary: Accept-Encoding x-content-type-options: nosniff x-download-options: noopen x-frame-options: DENY x-permitted-cross-domain-policies: none x-request-id: Z8uahE5gN39 X-Runtime: 0.012167x-Compatible: IE= Edge X-xs-protection: 1; Z8uahE5gN39 X-Runtime: 0.012167x-Compatible: IE= Edge X-xs-protection: 1; mode=blockCopy the code

Open the project configuration and you will find that Runner is online.

CI building is still a failure

Continue to run the CI pipeline in GitLab Runner, and you will see that an error still fails to pass the build.

Running with gitlab-runner 12.2.0~ bet.1803. G41d5c6ad (41d5C6AD) on Ci-runner S8sY8o6A Using Shell Executor Running with gitlab-runner 12.2.0~ bet.1803. Running on ci-runner... Fetching changes with git depthset to 50...
Reinitialized existing Git repository in /data/runner/builds/S8sY8o6A/0/project/.git/
> GitLab: The project you were looking for could not be found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
ERROR: Job failed: exit status 1
Copy the code

Remember the Authorization request headers and HTTP RFC specifications mentioned earlier? GitLab Runner uses an HTTP protocol such as https://gitlab-ci-token:[MASKED]@gitlab.domain/ rebo.git/to process CI tasks. The username and password in the request are incompatible with the Nginx ProxyPass field.

Using SSH instead of HTTP might solve the problem.

Try using the SSH protocol

Unfortunately, GitLab Runner is not officially supported to use SSH protocol for warehouse download. There are quite a few users with similar needs. If you are willing to find, there are quite a few issues like the following:

  • Supports git clone via ssh
  • Allow Configuration of how cloning is performed by runner/pipeline

Although this function is not provided directly by the authorities, there is a clone_URL in the official documentation. Looking up the code, we found that the implementation is very simple and only needs a slight modification to meet our needs:

// GetRemoteURL checks if the default clone URL is overwritten by the runner
// configuration option: 'CloneURL'. If it is, we use that to create the clone
// URL.
func (b *Build) GetRemoteURL() string {
	cloneURL := strings.TrimRight(b.Runner.CloneURL, "/")

	if! strings.HasPrefix(cloneURL, "http") {
		return b.GitInfo.RepoURL
	}

	variables := b.GetAllVariables()
	ciJobToken := variables.Get("CI_JOB_TOKEN")
	ciProjectPath := variables.Get("CI_PROJECT_PATH")

	splits := strings.SplitAfterN(cloneURL, ": / /", 2)

	return fmt.Sprintf("%sgitlab-ci-token:%s@%s/%s.git", splits[0], ciJobToken, splits[1], ciProjectPath)
}
Copy the code

The modified code is as follows:

// GetRemoteURL checks if the default clone URL is overwritten by the runner
// configuration option: 'CloneURL'. If it is, we use that to create the clone
// URL.
func (b *Build) GetRemoteURL() string {
	cloneURL := strings.TrimRight(b.Runner.CloneURL, "/")

	if! strings.HasPrefix(cloneURL, "http") && !strings.HasPrefix(cloneURL, "ssh") {
		return b.GitInfo.RepoURL
	}

	variables := b.GetAllVariables()
	ciJobToken := variables.Get("CI_JOB_TOKEN")
	ciProjectPath := variables.Get("CI_PROJECT_PATH")

	splits := strings.SplitAfterN(cloneURL, ": / /", 2)


	if strings.HasPrefix(cloneURL, "ssh") {
        ciProjectPath = strings.TrimLeft(ciProjectPath, "/")
        return fmt.Sprintf("git@%s:/%s.git", splits[1], ciProjectPath)
    }


	return fmt.Sprintf("%sgitlab-ci-token:%s@%s/%s.git", splits[0], ciJobToken, splits[1], ciProjectPath)
}
Copy the code

If you don’t want to waste time fiddling with your build environment, check out my previous post on compiling GitLab Runner from source code.

CI execution succeeded

When you execute the CI task again, you will find that it has progressed smoothly.

Running on ci-runner...
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /data/runner/builds/S8sY8o6A/0/project.git/
Checking out 860587f6 as master...
Skipping Git submodules setup
Warning: Permanently added '192.168. 7.0.x.x'(ECDSA) to the list of known hosts. From gitlab.domain:project 3957006.. 860587f master -> origin/master Updating 3957006.. 860587f Fast-forward .gitlab-ci.yml | 6 +++--- README.md | 1 + 2 files changed, 4 insertions(+), 3 deletions(-) Stopping service ... Stopping service ...done
Removing service ...

Removing service ... done
Network service is external, skipping
Creating service ...

Creating service ... done
Job succeeded
Copy the code

The last

GitLab will stop here for now, and maybe write a more comprehensive overview later when there is time. If you are interested in GitLab, check out my previous articles about it.

– EOF

I now have a small toss group, which gathered some like to toss small partners.

In the case of no advertisement, we will talk about software, HomeLab and some programming problems together, and also share some technical salon information in the group from time to time.

Like to toss small partners welcome to scan code to add friends. (Please specify source and purpose, otherwise it will not be approved)

All this stuff about getting into groups