Google Git-Repo Multi-repository project management

preface

After project modularization/componentization, each module is separated from the main project as a separate Git repository, and each module manages its own version. In a normal Android project, the separate sub-module repositories are managed through Maven repositories and then dependent on the main project in the same way that third-party libraries are introduced. The problem with project iterations in this state is the frequent release of submodule versions and the need to submit modified versions to the Merge Request into the main module, especially in the case of rapid iterations like our weekly release. Another problem is that Debug/ breakpoints can be inconvenient because the source code for the submodule may not be sent to Maven.

After working in the above state for some time, you need to find a solution to this problem. Try to try in the direction of multi – warehouse management. There are several management tools that support Git multi-repository: Git-repo, Git-submodule, gitslave, and Git-subtree. In addition to git-repo, several submodules either need to manually specify the local repository path, or the main repository and sub-repository are contaminated and not easy to operate. So I ended up using Google’s Git-repo (repo)

Git-repo

Repo is a code base management tool that complements Git (and can be huge), simply a set of script commands developed on top of Git using Python. Currently, the entire Android project (AOSP) is managed through repo. The latest version of the repository is about 700, so repo can support our existing projects in terms of code management and version management of multiple repositories. The process of migrating using a Repo is described and explained below.

Environment to prepare

Prepare the ladder and install the repo into the system:

Mkdir ~/bin PATH=~/bin:$PATH =~/.zshrc or ~/.bashrc or /etc/profile https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo chmod a+x ~/bin/repo
#The lastsourceSub-environment variable
Copy the code

Then you can manipulate the project, such as initializing the AOSP project:

Repo init -u https://android.googlesource.com/platform/manifest - b android - 4.0.1 _r1Copy the code

Manifest

The core of Repo management is Manifest. Every complex multi-repository project managed by Repo needs a corresponding Manifest repository, such as AOSP’s Manifest, which stores the configuration information of all its sub-repositories. The repo also reads the repository’s configuration file for administrative operations. The configuration is the structure defined by XML. There are two main configurations: the repository address (remote) for the sub-repository and the detailed sub-repository configuration information (Project). Here are some examples:

<?xml version="1.0" encoding="UTF-8"? >
<manifest>
  <remote  name="remote1"
           alias="origin"
           fetch=".."
           review="https://android-review.googlesource.com/" />

  <remote  name="remote2"
           alias="origin"
           fetch="[email protected]:group2/"
           review="https://android-review2.googlesource.com/" />         

  <default revision="master"
           remote="remote1"
           sync-j="4" />

  <project path="build/make" name="platform/build" groups="pdk" >
    <copyfile src="core/root.mk" dest="Makefile" />
    <linkfile src="CleanSpec.mk" dest="build/CleanSpec.mk" />
  </project>
  <project path="build/blueprint" name="platform/build/blueprint" groups="pdk,tradefed" revision="other_branch" remote="remote1"/>

<! -... -->

</manifest>
Copy the code

Explain the common nodes:

remote

Remote warehouse address configuration, can be multiple.

  • Name: name, also used for the name of a sub-repository (remote in.git/config)
  • Alias: indicates the alias, which can be omitted. You are advised to set it tooriginSet the git remote name of the sub-repository to this name, so that different names can be set to generate the same remote name
  • Fetch: Indicates the warehouse addressThe prefix, that is, the warehouse address of project is:remote.fetch + project.name
  • Pushurl: usually omitted, omitted, directly use fetch
  • Review.review.review.review.review.review.review.review.review.review.review.review.review.review.review.review.review.review.review.review.review.review.review.repo
  • Revision: Uses the default branch of this remote

The fetch has a pit, [email protected]:group2/, so the address starting with git scheme is not correct (SSH/HTTPS is normal), The final local remote address of the sub-repository is always the FETCH of the manifest address plus the name of the project (manifest.git_path + remote.fetch + project.name). The manifest_xml.py code in git-repo is found to be faulty. We need to fix the manifest_xml.py code by simply commenting out the 78 lines and then pushing them to our own repository to specify the system environment variables:

REPO_URL=your_fix_git-repo_git_url
#The address of this variable is used by the repo script
Copy the code

project

Subproject warehouse configuration, can be multiple.

  • Path: Path of the subrepository folder relative to the root directory during repo sync synchronization
  • Name: Git repository name of the sub-repository
  • Group: group
  • Revision: The name of the branch used
  • Clone-depth: depth of the repository synchronization with Git
copyfile

Child node properties of project.

  • SRC: relative path under project
  • Dest: indicates the relative path under the root path of the entire repository
linkfile

The child node property of project is similar to copyfile, except that the copyfile is changed to create the link file.

default

The default configuration used by project when no properties are set. Remote and Revision are commonly used.

other

Others are currently used less, see the official documentation for detailed and complete manifest format. Once you understand the concept of the Manifest configuration repository under Repo, you can create it for your own project, Just do it!

local_manifest

The local_MANIFEST is simply a local configuration that has a higher priority than the manifest that was set in the repo init. This is usually used when the remote manifest configuration is not changed and you want to set it to a local specific configuration. For higher versions of repo, create a configuration file in the working root directory:.repo/local_manifests/local_manifest.xml

If you want to overwrite the existing project configuration in the main manifest file, you need to remove-project first.

fatal: duplicate path project_name in .repo/manifest.xml
Copy the code

fix :

<remove-project name="project_name" />
<! -- rewrite project configuration again -->
<project path="xxx" name="xxx" revision="xxxx" remote="xxxx"  />

Copy the code

Repo command

Repo init -u yout_Manifest_git_URL After initializing your project’s repo workspace, repo sync, you can enter the normal feature development state. So let’s take the example of developing a feature, in order to illustrate the common commands that need to be used, which is the Repo Workflow I’m trying to use now.

repo init

Initialization.

repo init -u your_project_git_url
Copy the code

Other common optional parameters:

  • -b Indicates the selected manifest warehouse branch. The default is master
  • -m Specifies the default configuration file. The default configuration file is default.xml
  • — Depth =1 Git Clone depth, typically available when packaging on Jenkins, to speed up code clone
  • –repo-url=URL Use custom Git-repo code, such as the bug-fixed git-repo mentioned earlier
  • –no-repo-verify does not verify the source of the repo. This is usually added if the repo URL is custom

repo sync

After initializing the repo working directory, the next step is to synchronize the code:

repo sync
Copy the code

After synchronization is complete, each repository and its corresponding directory configured in the manifest will be generated in the current directory. Other common optional parameters:

  • -f does not block synchronization. That is, if a project fails, the next project continues synchronization
  • –force-sync Force overwriting the existing Git directory to point to a different object directory if needed. This operation may cause data loss
  • -d rolls the project back to the branch configured in the manifest
  • -m Manually specifies the manifest file to be used for the current operation
  • -t Uses the manifest file in the corresponding tag

After synchronization, you can open the project with your IDE. At this point, you will need to address the dependency of the subrepository in your project based on the current subrepository directory configuration. For example, for an Android project, you will need to include the subrepository in settting.gradle based on the project directory of the repo repository.

repo start

From the branch specified in the manifest configuration file, create a new branch for development, such as:

repo start your_feature_branch_name --all | [project1 project2]
Copy the code

If you find changes to other module repositories, start another branch of the corresponding project

Work in progress

Common commands to use during development:

  • Repo status: Similar to Git Status, it lists the current state of the Repo workspace
  • Repo diff: Same as Git diff
  • Repo forall <PROJECT_LIST> -c: Execute the command under (all) subrepositories. For example, repo has no similargit stashThe forall command can be implemented:repo forall -c git stash
  • Repo prune: Deletes the merged branch
  • Repo stage: Add files to the index table (staging area)
  • Repo manifest: displays the manifest information currently in use

repo upload

Commit code to Gerrit Code review. If you are not using Gerrit, you can either manually push a single repository as before or use repo forall -c Git push XXX

Other

For more detailed references, see the Repo Command Reference

Workflow

The sequential process of repo commands described in the previous section can be described as a common repo workflow. The development process given by AOSP can be seen first: Development

Here is a diagram of the repo-Workflow currently being attempted, as described in the previous section:

Specific to each repository, the original Git workflow location of the master repository does not change, each sub-repository added dev, build branch, then the corresponding version of the package and release package corresponding to the Manifest fixed branch configuration on the line, if there is no project deletion, Then the submodule repository version is also left untouched when compiling and distributing. Avoid frequent version, each module version of the frequent/tedious modification of the merger.

Jenkins

To use Jenkins, you need to use the Repo plug-in, which is also quite convenient, slightly migrating from the original Git to the Repo configuration. With Jenkins’ automatic packaging, add some automatic packaging configuration, such as remote parameterized build, custom Gradle packaging plugin, etc. If you need to use Jenkins Feature package, There is no need to create a new Feature branch in the Manifest project. So in general, the workload of the version is reduced a lot ~

End

Reference:

AOSP document

Git multi-project management

git-repo README