This article first in my blog, if useful, welcome to like collection, let more friends see.

The original author: William Kennedy | : Modules Part 01: according to the And I

Recently, I was trying to put together an article on the history of Go package management, hoping to deepen my knowledge of this area. When collecting information, I found this article and translated it.

This article, the first in a series, covers the basics of package dependency management. The paper addresses three pain points in Go development: how to solve the problem of only developing in the specified path of GOPATH, how to achieve effective version management, and how to support Go native toolset dependency management. For each of these, the Go Module provides corresponding solutions.

Judging from the content of the first article, the author is looking forward to a detailed analysis of the module mechanism of Go in the following articles. In other words, I always feel that the translation of this article is a little awkward. When I check it, I find several semantic misunderstandings, which is embarrassing.

The text of the translation is as follows:

introduce

Go Module is a comprehensive solution that Go provides for package dependency management. Since the initial release of Go, Go developers have raised three pain points about package management.

How to implement code development outside the GOPATH workspace;

How to implement dependency versioning management and identify compatibility problems of using dependencies effectively;

How to implement dependency management through Go native tools;

With the release of Go 1.13, all three issues have been addressed. Over the past two years, the Go team members have put a lot of effort into this. This article focuses on the change from GOPATH to the module mechanism and what problems modules solve. I will show you how the module works in language that is easy enough to understand.

I think it’s important to understand why modules work the way they do.

GOPATH

GOPATH is used to specify the physical location of the Go workspace and has served Go developers well. But it’s not friendly to non-GO developers, and it’s impossible to do Go development anywhere, anytime without any configuration.

The first problem the Go team had to solve was to allow the Go source repository to be clone anywhere on disk, not just in the workspace specified by GOPATH. And the Go toolset must still be able to successfully locate, compile, build, and test them.

The image above shows a Github repository, Ardanlabs /conf, which has a single package that provides support for application configuration processing.

Previously, if we wanted to use this package, we would download a copy to your GOPATH by going get and specifying the normalized name of the repository implementation. The normalized name of the repository is composed of two parts: the base URL of the remote repository and the repository name.

An example, in the Module before, if you Go get github.com/ardanlabs/conf, the code will be clone to $GOPATH/src/github.com/ardanlabs/conf directory. Based on GOPATH and repository names, the Go toolset always finds the correct location of the code no matter where we set up our workspace.

Import the parsing

Listing 1

Github.com/ardanlabs/c…

package conf_test

import(..."github.com/ardanlabs/conf"...).Copy the code

Listing 1 shows a snippet of code for importing other packages in the test file conf_test.go in the conf package.

When the test package name is _test, it means that the test code and the code under test are in different packages, and the test code must import the external code to be tested. In the code snippet above, you can see how the test code imports the CONF. Based on the GOPATH mechanism, it is very easy to resolve the path of the imported package. The Go toolset can then successfully locate, compile, and test the code.

What if GOPATH doesn’t exist or the directory structure doesn’t match the repository name?

Listing 2

import "github.com/ardanlabs/conf"

// GOPATH mode: Physical location on disk matches the GOPATH
// and Canonical name of the repo.
// GOPATH pattern: The physical location of the disk matches the specification names of GOPATH and repository
$GOPATH/src/github.com/ardanlabs/conf

// Module mode: Physical location on disk doesn’t represent
// the Canonical name of the repo.
// Module mode: The physical location on the disk does not necessarily match the full name of the warehouse.
/users/bill/conf
Copy the code

Listing 2 shows what happens if you clone the repository to any location. When developers choose to download the code wherever they want, it is not possible to import the package name to resolve the actual location of the source code.

How to solve this problem?

We can specify a special file to use to specify the specification name of the repository. The location of this file is understood as an alternative to GOPATH, where the repository’s canonical name is defined, and the Go tool can use this name to resolve the location of imported packages in the source code, regardless of where the repository has been cloned to.

We named this particular file go.mod and called the new entity defined in this file by the specification name Module.

Listing 3

Github.com/ardanlabs/c…

module github.com/ardanlabs/conf
Copy the code

Listing 3 shows the first line of the go.mod file in the conf repository.

This line defines the name of the module, which also represents the full repository name that the developer expects to use to reference any part of the library’s code. Now that it doesn’t matter where the library was downloaded, the Go toolset locates and resolves internal package imports based on module file location and module name, such as the conf package import in the test file in the previous example.

Now, the module mechanism allows us to download code anywhere. The next problem to solve is how to bundle the code together for version control.

Bundling and versioning

Most version management systems support tagging at any commit point. These tags are usually used to publish new features (v1.0.0, V2.3.8, etc.) and are generally immutable.

As you can see, conf has been tagged with three different versions. These three tags follow the format of the semantic version.

Using a version management tool, you can clone any version of the CONF package by specifying a tag. But there are two problems.

  • Which version of the package SHOULD I use;
  • How can I know which version of the package is compatible with the code I write or use;

Once these two questions are answered, a third question arises:

  • Where to download the dependent code, and the Go tool should be able to find and access it;

Then things got worse.

In order to use a particular version of the conf package, you must download all the conf dependencies. This is a common problem for all projects that have dependency delivery.

In GOPATH mode, all dependencies can be identified and downloaded using Go Get, and then placed in a workspace specified by GOPATH. But this is not a perfect solution, because Go Get can only download and update the latest code from the Master branch. When initially writing code, downloading code from the master is no problem. But after a few months, some of the dependencies may have been upgraded, and the latest code for the Master branch may no longer be compatible with your project. This is because your project does not follow clear version management, and any upgrade may bring an incompatible change.

In Module mode, downloading all dependencies to a single workspace via Go Get is no longer preferred. You need a way to specify a compatible version for each dependency throughout the project. It also supports the introduction of different major versions of the same dependency to prevent different major versions of the same package in a project.

The community has developed some solutions to these problems, such as DEP, GODEP, Glide, etc. But Go requires an integrated solution. This solution maintains these direct and indirect dependencies by version by reusing the go.mod file. Then, treat any version of the dependency as an immutable package of code. This particular version of the immutable code package is called a Module.

Integrated solutions

The figure above shows the relationship between the repository and the module. It shows how to refer to packages in a particular version module. In this case, the code in CONF-1.1.0 imports the CMP package from version 0.3.1 go-CMP. Since the dependency information is already in the CONF module (stored in the module file), Go can get the specified version of the module from the built-in toolset and build it.

Once modules are in place, many convenient engineering experiences emerge:

  • Support can be provided to Go developers worldwide, such as Build, retain, Authenticate, validate, fetch, cache, etc.
  • Build a proxy server in front of different version management systems to implement the previously mentioned support;
  • You can verify that a module has been modified, regardless of how many times it has been built, where or from whom you got it,

In this regard, it is very fortunate that the Go team has already provided a lot of support for this in Go 1.13.

conclusion

This article attempts to set the stage for a later discussion of what the Go module is and how the Go team designed the solution. There are still some issues to discuss, such as:

  • How is a particular version of the module selected?
  • What is the organizational structure of a module file and what options does it provide to help you control the selection of modules?
  • How is the module compiled, retrieved, and cached to a local disk to help parse imported packages?
  • How do I validate modules with semantic versions?
  • How and what are the best practices for using modules in your projects?

In the following articles, I plan to provide a deeper understanding of these issues. Now, make sure you understand the relationships between repositories, packages, and modules.