A true code leak story

It happened in the era of feature phones. Our project team was developing a mobile phone, and there were about 20 software developers. As a result, a week after the phone’s launch, another small manufacturer came out with a phone with a software interface that did almost everything except boot up.

Because at that time, everyone was almost using MTK, Qualcomm provided solutions, are unified menu type functions, you can’t come up with strong evidence that someone stole your code.

It was later found out internally that a developer had indeed leaked the code, and USB ports on all computers were subsequently banned.

This is A true story from my own experience, when everyone was responsible for A module, such as: A was responsible for call management and phone book, B was responsible for system Settings, C was responsible for SMS and MMS… When compiling, it is necessary to put all the code together, unified compilation, which means that all the software personnel can get all the source code, which also buried hidden trouble for code leakage, this serious incident, after all, people die for money, birds die for food!

So, are there any code controls that address this permission issue?

In the current project, it is emphasized to be layered and divided into modules, which is considered from the perspective of software engineering. If you take it a step further and divide these modules into smaller subsystems, each developer is responsible for his own module and can only pull his own code, so he can’t get all the modules in a project.

Only the project integrator (administrator) has full authority to pull all the source code to build the entire system, so that you have better control over code security issues.

To achieve such code control, using the Git tool submodule can be completed, this article, we will explain the use of Git submodule in detail.

This article is tool-based and may be quite long. In order to provide one-stop service, I will record relevant resources, steps, error information and other details for future reference.

Anyway, after this article, you can learn and understand the following points:

  1. Basic usage of Jenkins;
  2. Git submodule basic instruction usage;
  3. Through three demo projects, step by step to achieve code security control;
  4. Using Jenkins + Git submodule to realize automatic compilation;
  5. Git submodule and subtree

If you need any of the software and code resources mentioned in this article, download them at the end of the article.

Ii. Basic use of Jenkins

1. What is Jenkins?

Jenkins is an open source continuous integration tool written in Java, which means it helps us automate all kinds of projects. Jenkins runs in Servlet containers (such as Apache Tomcat) and can be installed in Ubuntu with one click using apt-get.

Jenkins has the following characteristics:

  1. Embedded in the Web server, through the browser to operate, very convenient;
  2. You can execute projects based on Apache Ant and Apache Maven, as well as arbitrary Shell scripts and Windows batch commands;
  3. A build can be triggered by various means. For example, when a submission to version control is triggered, it is scheduled through a Cron-like mechanism, and requests can be made from a specific URL when other builds have been completed;
  4. Jenkins’ powerful plug-in enables Jenkins to integrate many software, which may help us to continuously integrate our engineering projects.
  5. Gives users a lot of authority and flexibility to automatically publish, deploy, and so on.

I’m not going to blow on the other ones. I think they work very well. If you have a chance, you can try them. In addition, THE VIRTUAL machine I tested was newly installed Ubuntu16.04-64. Follow the following procedures to ensure smooth operation.

There is a lot of information on the Internet about how to install JDK and Jenkins, and some of the procedures are questionable, or some of the key steps are not clearly written. I’m going to write it down here so that you can do it right once.

If you’re already familiar with the installation process, you can just slide on to the next topic.

2. Install JDK8

(1) Download and decompress

Download JDK -8u221-linux-x64.tar.gz, unzip it to /home/sewain/openSource.

sudo tar -zxvf jdk-8u221-linux-x64.tar.gz -C /opt
Copy the code

(2) Set environment variables

Execute the command vim ~/.bashrc and add the following at the end:

Export JAVA_HOME=/opt/jdk1.8.0_221 export PATH=$JAVA_HOME/bin:$PATH export CLASSPATH=./$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jartCopy the code

I recommend that you use these environment variables as well. If you upgrade JDK versions in the future, you only need to change JAVA_HOME.

(3) reload environment variables

Execute the command source ~/.bashrc and the environment variable takes effect.

Verify that Java -version is OK if the following information is displayed:

Java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-B11) Java HotSpot(TM) 64-bit Server VM (build 25.221 - bl1, mixed mode)Copy the code

3. Install the Jenkins

(1) Import the GPG key of Jenkins repository

sudo wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
Copy the code

The command above should output OK, which means that the key has been imported successfully and that packages from this repository will be considered trusted.

(2) Jenkins repository is added to the system

sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
Copy the code

(3) Use APT installation

sudo apt update
sudo apt install jenkins
Copy the code

The Jenkins service will start automatically after the installation process is complete, which can be verified by the command systemctl Status Jenkins.

systemctl status jenkins

(4) Configure ports

Jenkins is embedded in a Tomcat server and uses port 8080 by default, which is easy to conflict with other services, so it needs to be modified. Two files are involved:

File 1: /etc/init.d/ Jenkins

In the PATH variable on the first line, add your own JDK address:

PATH = / bin: / usr/bin, / sbin, / usr/sbin, / opt/jdk1.8.0 _221 / binCopy the code

File 2: /etc/default.jenkins

Change the value of HTTP_PORT to the new port number, for example, HTTP_PORT=9090.

(5) Jenkins’ start and stop instructions

sudo service jenkins start
sudo service jenkins stop
Copy the code

If you encounter an error, you can use these two instructions repeatedly to troubleshoot the error.

4. Configure Jenkins in the browser

Enter HTPP ://localhost:9090 in the browser, wait a moment, the interface appears:

According to the prompt interface, from the/var/lib/Jenkins/secrets/initialAdminPassword file copy the security code (root), to fill in the browser window.

The plug-in installation window appears. Generally, you only need to install the recommended plug-in:

After the plug-in is installed, the administrator configuration page is displayed:

With that, Jenkins’ installation is complete!

5. Configure a small project in Jenkins

(1) Prepare a test code Test1

Jenkins is just a component framework, and the specific compilation process is up to the user. Jenkins first uses git tools to retrieve the code from the remote repository and then executes user-specified compilation instructions.

Therefore, we need to prepare a copy of the test code in advance and put it in a remote repository that Jenkins can access. Of course, it is also possible to deploy a Git repository on your local Ubuntu system. For convenience, the code I tested, Test1, is in Gitee.

(2) Create a new project

(3) Enter the project name and select the first Free project.

(4) Under the first tag General, enter the Description of the project.

You can write anything you want.

Source Code Management

Jenkins needs to obtain the source code during construction (compilation), so you need to configure the address and account information (username and password) of the Git repository.

First, under the Add drop-down button, select Jenkins:

Enter Username and Password:

After the account is added, select the Gitee account from the drop-down list. Then, you can see that the red permission error message disappears, indicating that the source code in the remote repository can be pulled.

(6) Build Triggers Triggers selection

You can choose different ways to trigger as needed, such as timing, when another project is successfully built, and so on.

We don’t select any items here, now we will manually click the button on the main screen to trigger.

(7) Build Environment

I used this part less and used other tools to assist Jenkins’ function.

(8) Build compilation

It is to tell Jenkins how to build the system, that is to say: Jenkins is only an automated build system, and the specific compilation process can be decided by the users themselves, including the following selection items:

We choose Execute Shell directly and enter the following command:

(9) Post-build Actions

Tell Jenkins: After compiling a project, what do you need to do? For example, sending an email, triggering automatic compilation of the next project, etc., and you can add multiple actions. The options are as follows:

After the above steps are configured, Save, then you can see the full picture of the project in the main interface, as shown below:

In Jenkins backstage, save the path of this project is: / var/lib/Jenkins/jobs/Test1.

6. Manually trigger compilation once

Since we did not select any trigger condition in step (6) above, we need to manually click the Build Now button on the left in the main screen of project Test1 to trigger it.

In the Build History column on the left, you can see the Build History. Click the number of a Build record to see the details of this Build.

In the compile details, click the Console Output button on the left to see the compiled Output: the executable compiled successfully.

We can see in Jenkins background, source code to be pulled to/var/lib/Jenkins/jobs/Test1 / workspace directory:

At this point, you’ve learned the basics of Jenkins!

Git submodule: Git Submodule: Git Submodule

Basic use of Git submodule

1. What is a Git submodule?

Git submodule is a tool for multi-module management that allows one project to be used as a repository and other projects to exist as sub-modules in the parent project.

The parent project and the subproject are submitted separately, that is, the information submitted by the parent project only contains the information of the subproject, not the code of the subproject; Subprojects have their own separate commit, push, and pull operations.

Git submodules are usually used in large projects, and often need to be divided into several sub-projects for code management for ease of reuse or for code security.

Common instructions include:

Git submodule update git submodule update git submodule init Recursively clone the entire project: Git clone –recursive git submodule foreach git pull

2. Use three small projects to test the use of subModule

For demonstration purposes, let’s create three projects and push them all to a remote repository, using Gitee.

  1. Test1: compile a dynamic library: libtest1.so;
  2. Test2: compile to get a dynamic attitude: libtest2.so;
  3. Test3: Compile to produce an executable program that loads and calls functions from the two dynamic libraries above.

Why modules are designed this way: Security!

  1. Developer A: Is responsible for Test1 and has no access to other modules’ code;
  2. Developer B: responsible for Test2 and has no access to other modules’ code;
  3. Project manager: responsible for Test3 and code integration, have access to all code;

The project manager needs to add Test1 and Test2 as sub modules to Test3 by executing the following command:

Git submodule add gitee.com/ /test… Test1 git submodule add gitee.com/ /test… test2

After adding Test1 and Test2 as submodules to Test3, see what happens to the file:

You can also take a look at the.gitModules file to see how git tools manage submodules through this configuration file.

The Test3 directory contains a script called build.sh, which contains the following instructions:

Just call the make command, execute it, and output:

At this point, we are done adding submodules.

3. Compile and verify in an empty directory

Clone the test1 and test2 folders from another empty directory, and find that the test1 and test2 folders are empty as shown below:

To manually retrieve all submodules, execute the command:

git submodule update --init --recursive
Copy the code

Test1 and test2 are pulled down using the tree command. There is one problem to note: the submodule’s head does not point to the master branch, although the submodule’s code is pulled off. This needs to be handled manually, as shown in the figure below:

At this point, we will execute the build.sh script again in the test3 directory, and all submodules will compile smoothly.

For the above steps, we manually test the subModule compilation process in a local temporary directory.

In the next section we will deploy this process to the Jenkins system, so the instructions we just executed will need to be written in the build.sh script. The build.sh content becomes:

4) In Jenkins, use Git Module to compile all modules

The following operations are performed in the Jenkins panel of the browser.

1. Reconfigure the project

Since we are compiling the entire project in Test3 (Test1 and Test2 are included as submodules), we first remove the previously added Test1 project, as shown in the figure below:

Then re-add project Test3 and review the steps:

  1. Enter description information and select free style items.
  2. Enter the git repository address and account information and select the Master branch.
  3. Trigger not set;
  4. The compile environment is not set.
  5. Compile: Select Execute shell to Execute the script and enter the compilation command./build.sh. (As mentioned earlier, Jenkins is an automated build framework, and the specific build process is determined by the user, so we will execute the build.sh script in Test3.) ;
  6. Action is not set after compilation.

Of course, you can make changes directly from the previous Test1 project.

At this point, we click the Build Now button directly in Jenkins, and if nothing goes wrong, we will get a Build error (red error circle below Build History on the left).

Click the Console Output button to view the Output.

As mentioned above, the head does not point to the master branch after the submodule is acquired, so we need to change it manually during the first compilation (I can’t find any other way, if you know, please comment!).

Manual solution: in the command window, enter the Jenkins system Test3 directory/var/lib/Jenkins/jobs/Test3 / workspace, perform the following a few instructions:

git submodule update --init --recursive
cd test1/
git checkout master
cd -
cd test2/
git checkout master
cd -
Copy the code

At this point, re-trigger compilation, can be successful!

Five, the summary

This article is tool-based, and once deployed, it will only take a click of a button in the browser to compile each time, rather than SSH to a remote computer to do it manually.

Here are a few things to look into if you want to dig a little deeper:

1. How does Jenkins maintain his build history

In the directory/var/lib/Jenkins/jobs/Test3 / builds below, you can see a lot of digital named folder, compile information recorded every time.

2. Action after compilation

In our build.sh compilation script, we just generated the executable file, and you can continue to extend features such as automatic deployment.

Or in the post-build Actions of the project configuration, rewrite a script file specifically for automatic deployment.

3. Git subtree

It’s similar to git submodules, but different in nature.

  1. Subtree directly copies the submodule code to the main module, which is easy to use.
  2. Submodule uses “Pointers” to point to submodules, and commands are more complex and powerful.

4. Continue to study Jenkins’ plugins

Download resources

The resources used in this article are placed in the web disk. If necessary, please send the number 030 in the message area of the public number to receive the download address.


If your public number would like to reprint this article, please private message or message, I give you open long white.





【C language 】 C language pointer – from the underlying principle to the tricks, with graphics and code to help you explain thoroughly the original GDB underlying debugging principle so simple step by step analysis – how to use C to achieve object-oriented programming to improve the code forced lattice sharp weapon: Macro definition – from the beginning to the abandonment of setJMP and LongjMP in C language, to achieve exception capture and coroutine

Internet of Things gateway development: Design process based on MQTT message Bus (1) Internet of Things gateway development: The design process based on MQTT message bus (below) my favorite way of communication between processes – message bus

[Operating system] Why spacecraft, missiles like to use microcontroller, rather than embedded system?

[Internet of Things] All that encryption and certification stuff goes deep into the LUA scripting language and gives you a thorough understanding of how debugging works

Based on my failed career experience: A few tips for budding techies