The author | Dong Jianhui, ShengAoFei source | alibaba cloud native public number

The problem background

The problem is that we want to make a change to the Module path of Dubo-Go, using dubbo.apache.org/dubbo-go/v3 instead of github.com/apache/dubbo-go.

First, we did a path map and placed a Dubbogo /v3 file under dubbo.apache.org, which looks like this:

<html> <head> <meta name="go-import" content="dubbo.apache.org/dubbo-go/v3 git <https://github.com/apache/dubbo-go>"> < meta name = "go - the source" content = "dubbo.apache.org/dubbo-go/v3 git {/ dir} < https://github.com/apache/dubbo-go/tree/3.0 > {< https://github.com/apache/dubbo-go/blob/3.0 / dir} / {file} {line} # L > "> < meta HTTP - equiv =" refresh "content =" 0; url=https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3"> </head> <body> <p>Redirecting to <a href="<https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3>">pkg.go.dev/dubbo.apache.org/dubbo-go/v3</a>... </p> </body> </html>Copy the code

Second, we changed the module of go.mod and all corresponding imports, and changed the module path used by all submodules referencing Dubo-go.

Problem analysis

After completing the above modifications, WE found CI failed when we proposed PR. After further log investigation, we determined that IT was CI that made an error when running integration test. The specific error message is as follows:

The execution logic of this section is to use Docker to build an image of the integration test content in Dubo-Go and start the container for testing. The images used in the packaging Dockerfile path in github.com/apache/dubbo-go/test/integrate/dubbo/go-server directory, controlled STEP identification error log, We can locate the specific error in the following two steps:

#... # STEP 9 RUN test ${PR_ORIGIN_REPO} && go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/{PR_ORIGIN_REPO}@${PR_ORIGIN_COMMITID} || go get -u dubbo.apache.org/dubbo-go/v3@develop # ... # STEP 11 RUN go mod tidy && go install github.com/apache/dubbo-go/test/integrate/dubbo/go-serverCopy the code

In STEP 9, we used go mod Edit-replace to replace the dependency path of DubboGo with the repository address and commit ID of the PR request. On this basis, an error occurs when the mirror build goes to STEP11 and tries to use the go mod tidy pull package.

Looking back at the error log, we can see:

Since we only specified the COMMIT ID, when the Go Mod actually runs, it will generate a hypothetical version number for us (see the issue extension at the end of this article for more on the hypothetical version number) that captures the latest valid tag of the remote repository as V1.4.1. The remote repository here is github.com/Mulavar/dubbo-go. This is my own Dubo-Go branch, which pulled the fork early and ended up with a tag of V1.4.1 】, which has been incremented to v1.4.2, and the main version is V1. But in previous, our dubbogo module path is declared as dubbo.apache.org/dubbogo/v3, major version is v3, this led to go mod edit – before and after the replace rely on major version is v1 and v3 respectively, An inconsistency occurred and an error occurred while actually pulling the Dubbogo dependency.

Problem solving

In the problem analysis section, we locate this problem in STEP 9 of mirror construction, namely:

# STEP 9
RUN test ${PR_ORIGIN_REPO} && go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/{PR_ORIGIN_REPO}@${PR_ORIGIN_COMMITID} || go get -u dubbo.apache.org/dubbo-go/v3@develop
Copy the code

In this step, we used go mod edit-replace to replace a V3 version of the Go module with a V1 version of the module, resulting in the main version inconsistent error. Therefore, we only need to specify the main version of the replaced module to be v3 when replacing the dependency path. We also add the v3 suffix to the module dependency path after Go mod Edit-replace, as shown below.

Modify before:

go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/{PR_ORIGIN_REPO}@${PR_ORIGIN_COMMITID}

Tail Modified:

go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/${PR_ORIGIN_REPO}/v3@${PR_ORIGIN_COMMITID}

After the problem was fixed, we submitted the code to check CI. According to the log information printed in STEP 8, we can see that v3 is added to the replaced Dubbogo path, and the main version (V3.0.0-20210509140455-2574eAB5AD0b) is also V3 when we pulled the package. Successfully pulled dependencies.

Problems develop

1. Semantic Import Versioning

When we use The Go Modules to build the dependencies of the project, the dependencies of the Go project need to be declared with the version number we use. Considering that compatibility is always the most important and headache for developers every time they release a new version, Therefore, GO standardizes version numbers through Semantic Import Versioning ** to ensure that each developer can specify dependent versions to use according to their project needs. According to go’s semantic import versioning guidelines, the version number consists of three parts:

Note: The above figure and some of the content refer to the article “Go Modules”.

Major version indicates a new Major version, even though the new version may not be compatible with the old version.

The Minor version indicates that this is an iteration in a large version, which is mainly used to increment when adding features.

Patch version is the most fine-grained version update, indicating some bug fixes.

There are two ways to specify the major version, one is the above direct declaration, the other is to add the version suffix at the end of the project path, such as dubbogo 3.0 declaration module is dubbo.apache.org/dubbo-go/v3, where v3 is a major version declaration.

2. pseudo-version

Go encourages us to specify explicit version numbers when using dependencies, such as the dependency on cloud.google.com/go declared in Dubbogo’s go.mod:

But considering that there are scenarios where we want to use an unpublished version of a module dependency (which only has a known COMMIT ID), such as the github.com/Microsoft/go-winio dependency declared by Dubbogo, You can use a hypothetical version number instead of the real version number. Assume the format of the version number is:

// (1) YYYYMMDDHHMMSS - ABCdef123456 // (2) Vx.y.(Z+1)-0. vX.Y.(Z+1)-0.yyyymmddhhmmss-abcdef123456+incompatible // (4) vX.Y.Z-pre.0.yyyymmddhhmmss-abcdef123456 // (5) vX.Y.Z-pre.0.yyyymmddhhmmss-abcdef123456+incompatibleCopy the code

You can see that pseudo-version is divided into three parts by short slashes. The first part X.Y.Z is the same as that mentioned in section 4.1, namely, major version, minor version, patch version. The second part is a timestamp in the yyyYMMDDHHMMSS format, and the third part is a 12-bit COMMIT hash that can be specified manually or generated automatically by GO if it does not exist.

For pseudo-version, the go generation rules are as follows:

  • Query the latest tag of the major version of the project (v1 by default if the dependent path of the project does not have an explicit v2/v3 suffix).
  • If there is a tag, use the latest incremental patch version of tag.
  • No tag, automatically generated according to the suffix version number of the project path band (such as v3 automatically generated 3.0.0pseudo-version).

Following this rule, and looking back at the previous problem analysis and problem solving, we can see why pseudo-version of Dubbo-Go was v1.4.2 in the first place, precisely because Go thought that dubbogo after replace depended on the main version being V1, We searched for the latest tag under the major version and added patch version, which caused the inconsistency between the previous and the major versions. When we added v3 to the version path, GO could not find the tag under the corresponding major version, so v3.0.0 was automatically generated for us, thus passing CI.

3. Go modules are nested

Unlike Java, go has no concept of submodules. Sometimes, though, we’ll see multiple Go modules in a repo, such as a project with a go.mod in its root directory and go.mod in some of its subdirectories.

This is called a nested module in Go modules, not a parent-child module, which means the two modules have nothing to do with each other and are independent of each other.

When do you need a single REPO with multiple modules? In general, there are two situations where we would consider using a single REPO with multiple modules.

  1. A nested module changes so frequently that it needs to be published often.
  2. The nested modules in the root module only depend on a fixed version.

In both cases, there is no strong version binding between the two modules, but some other reasons need to be placed under the same RPEO, so a single REPO with multiple modules is formed.

4. Content parsing of dubboGo static mapping files

Dubo-go implements module redirection using static file mapping, which looks like this:

The core of this is the meta tags Go-import and Go-Source.

<html> <head> <meta name="go-import" content="dubbo.apache.org/dubbo-go/v3 git <https://github.com/apache/dubbo-go>"> < meta name = "go - the source" content = "dubbo.apache.org/dubbo-go/v3 git {/ dir} < https://github.com/apache/dubbo-go/tree/3.0 > {< https://github.com/apache/dubbo-go/blob/3.0 / dir} / {file} {line} # L > "> < meta HTTP - equiv =" refresh "content =" 0; url=https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3"> </head> <body> <p>Redirecting to <a href="<https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3>">pkg.go.dev/dubbo.apache.org/dubbo-go/v3</a>... </p> </body> </html>Copy the code

1) go to import

Go-import tells Go Get where to find the source code. The content is divided into three parts:

  • dubbo.apache.org/dubbo-go/v3: Module declaration for this project.
  • git: Version control tool used.
  • <https://github.com/apache/dubbo-go>: Tell Go Get where to find the source code for this project.

2) the go – source

The function of go-source is to generate a specific Go doc (now pkG.go.dev) document for the project. There are four parts in total. The first two parts, like go-import, are the module declaration and version control tool for the project.

  • {/ dir} > < https://github.com/apache/dubbo-go/tree/3.0: declares the location of the project’s source code.
  • {< https://github.com/apache/dubbo-go/blob/3.0 / dir} / {file} {line} > # L: maps documents and code to help us jump to the corresponding content when we click on the document’s directory tree.

In https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3, for example, we click one of the file:

It jumps to the corresponding document for the corresponding file.

If you are interested in the Apache/Dubo-Go project, please search the Nacks group number 31363295 and join the Nacks Communication Group!

The resources

  • “The Go Blog – Using The Go Modules: https://blog.golang.org/using-go-modules
  • “Go Modules Reference: https://golang.org/ref/mod
  • “How Go Module launch v2 and above version” : https://blog.cyeam.com/go/2019/03/12/go-version
  • “Go Modules, rounding: https://www.sulinehk.com/post/go-modules-details/

Author’s brief introduction

Github @Mulavar dong Jianhui (Github @Mulavar), just graduated from VLIS Laboratory of Zhejiang University, is currently working as a development engineer of ape Counseling service.

Github @aofei, author of Goproxy. cn project, has submitted a lot of PR for go source code [such as mod tools].