Jenkins multi-branch pipeline automates building Android projects, packaging them and uploading them to fir.im

Jenkins multi-branch pipeline automated construction implements the management of all branches under the Git repository. Whenever the code of the target branch of the Git repository is updated, the packaging and uploading process will be automatically triggered. All branches need a common pipeline-written jenkinsFile and a personalized config.yaml configuration file to manage the automated build, package and upload of Git branches.

1. Prepare tools

1.JDK (8/11), SDK, Gradle, Git download and configure environment variables

2. Jenkin installation

Versions of Windows

The next step is to go straight to the installation. The browser opens http://localhost:8080/ to display the Jenkins home page.

(1) Global Tool Configuration project JDK,SDK,Gradle; When configuring Gradle environment variables, pay attention to a pit, Gradle version should be the same as the compiled project Gradle version, otherwise there will be SDK License problems;

② Install all BlueOcean plug-ins to facilitate graphical interface management

③ Install all recommended plug-ins

(4) Create the project, select the multi-branch pipeline, configure the git repository address and login credentials, set the construction parameters, etc

The workspaceDir of the Jenkins global project needs to be manually modified. Because the Jenkins installation path is too long in the system disk path, the error will be reported when the resources are not found. This is the longest file path in the Windows system. We need to change the workspaceDir path in Jenkins’ config. XML file, and then click Reload Configuration from Disk in Manager Jenkins to apply the Configuration

3. Understand PipeLine syntax

pipeline

The jenkinsFile determines the steps of the project build, such as whether the branches of the Git repository will trigger the automated build after the code is submitted, the packaging method, the execution of Gradle script tasks after the completion of the packaging, and the email notification of the build result, etc

4.Python

Install the Python requests library, which is a common HTTP request library written in Python. If you don’t have PIP installed, you can easily send HTTP requests. Install Properly Python on various platforms. You can use the following commands to install Properly Python: setuptools, PIP, virtualenv

pip install requests
Copy the code

This is in order for Jenkins to successfully execute the Python script and automatically upload apK to fir.im. This is in order for Jenkins to successfully execute the Python script and automatically upload APK to fir.im. This can only be done if Python is specified as the full path.

5. Fir. Im account

The API token is obtained and the APK interface is called. The fir.im domain name should be changed to the latest one

6. The pluginpipeline utility stepsPlug-in installation

To read the YAML file configuration in jenkinsFile, you can share a set of Jenkins files in the project and then build the APK based on the configuration in config.yaml

Compile JenkinsFile and config.yaml

Here is the JenkinsFile code for the Android Demo project:

Def loadValuesYaml(x){def loadValuesYaml = readYaml (file: 'config.yaml') return valuesYaml[x]; } pipeline {// Agent nodes are configured to execute Android builds, iOS builds only, and Go projects. // Agent {label 'Android'} agent {label 'Android'} Agent any options {// Time out, Options: Retry (3) Timeout (time: 1, unit: 'HOURS')} environment{// A set of global environment variable key value pairs used in stages BUILD_TYPE = loadValuesYaml('buildType')} stages {// Here we have the default checkout code to start the build and publish // the best way to configure the build parameters according to the branch is to get the corresponding configuration file from a YAML file stage('readYaml'){ steps{ script{ println MARKET println BUILD_TYPE } } } stage('Build master APK') { when { branch 'master' } steps { bat "./gradlew clean assemble${MARKET}${BUILD_TYPE}" } post { failure { echo "Build master APK Failure!" } success { echo "Build master APK Success!" } } } stage('Build dev APK') { when { branch 'dev-hcc' } steps { bat "./gradlew clean assemble${MARKET}${BUILD_TYPE}" } post { failure { echo "Build dev APK Failure!" } success { echo "Build dev APK Success!" }} stage('ArchiveAPK') {archiveArtifacts(artifacts: 'app/build/outputs/apk/**/*.apk', fingerprint: true, onlyIfSuccessful: true) } post { failure { echo "Archive Failure!" } success { echo "Archive Success!" }} stage('Report') {// Display submission information steps{echo getChangeString()}} stage('Publish'){// Publish fir.im steps{bat './gradlew apkToFir' } post { failure { echo "Publish Failure!" } success { echo "Publish Success!" Emailext Body: 'apK version updated ', Subject:' APK uploaded successfully ', to: '137**6*****@163.com'}}}}} //report @noncps def getChangeString() {MAX_MSG_LEN = 100 def changeString = "" echo "Gathering SCM Changes..." def changeLogSets = currentBuild.changeSets for (int i = 0; i < changeLogSets.size(); i++) { def entries = changeLogSets[i].items for (int j = 0; j < entries.length; j++) { def entry = entries[j] truncated_msg = entry.msg.take(MAX_MSG_LEN) changeString += "[${entry.author}] ${truncated_msg}\n" } } if (! changeString) { changeString = " - No Changes -" } return changeString }Copy the code

Here is the contents of config.yaml

market: Google
buildType: Debug

Copy the code

3. Write tasks in build.gradle under the module

In the Android {} block

 task apkToFir {
        //dependsOn 'assembleDebug'
        doLast {
            //def upUrl = "http://api.fir.im/apps"
            def upUrl = "http://api.bq04.com/apps"
            def appName = "jenkinsDemo"
            def bundleId = project.android.defaultConfig.applicationId
            def verName = project.android.defaultConfig.versionName
            def apiToken = "d319ac25103******dc4fbf545ad8a7"
            def iconPath = "app/src/main/res/mipmap-hdpi/ic_launcher.png"
            def apkPath = "app/build/outputs/apk/google/debug/app-google-debug.apk"
            def buildNumber = project.android.defaultConfig.versionCode
            def changeLog = Version Update Log
            // Execute the Python script
            def pythonPath = "c:\\users\\xinmo\\appdata\\local\\programs\\python\\python39\\python.exe"
            def process = "${pythonPath} upToFir.py ${upUrl} ${appName} ${bundleId} ${verName} ${apiToken} ${iconPath} ${apkPath} ${buildNumber} ${changeLog}".execute()
            println("Start uploading to FIR")
            // Get the Python script log for error debugging
            ByteArrayOutputStream result = new ByteArrayOutputStream()
            def inputStream = process.getInputStream()
            byte[] buffer = new byte[1024]
            int length
            while((length = inputStream.read(buffer)) ! =- 1) {
                result.write(buffer, 0, length)
            }
            println(result.toString("UTF-8"))
            println "Upload over"}}Copy the code

4. The Python script is executed to upload fir.im

Here is the code for uptofir.py:

# coding=utf-8
# encoding = utf-8
import requests
import sys


def upToFir() :
    Print the length of the passed parameter array for easy verification
    upUrl = sys.argv[1]
    appName = sys.argv[2]
    bundleId = sys.argv[3]
    verName = sys.argv[4]
    apiToken = sys.argv[5]
    print (apiToken)
    iconPath = sys.argv[6]
    apkPath = sys.argv[7]
    buildNumber = sys.argv[8]
    changeLog = sys.argv[9]
    print(apkPath)
    queryData = {'type': 'android'.'bundle_id': bundleId, 'api_token': apiToken}
    iconDict = {}
    binaryDict = {}
    # Get upload information
    try:
        response = requests.post(url=upUrl, data=queryData)
        json = response.json()
        iconDict = (json["cert"] ["icon"])
        binaryDict = (json["cert"] ["binary"])
    except Exception as e:
        print(e.message)

    # upload the apk
    try:
        file = {'file': open(apkPath, 'rb')}
        param = {"key": binaryDict['key'].'token': binaryDict['token']."x:name": appName,
                 "x:version": verName,
                 "x:build": buildNumber,
                 "x:changelog": changeLog}
        req = requests.post(url=binaryDict['upload_url'], files=file, data=param, verify=False)
        print(req.status_code)
    except Exception as e:
        print(e.message)

    # upload logo
    try:
        file = {'file': open(iconPath, 'rb')}
        param = {"key": iconDict['key'].'token': iconDict['token']}
        req = requests.post(url=iconDict['upload_url'], files=file, data=param, verify=False)
        print(req.status_code)
    except Exception as e:
        print(e.message)


if __name__ == '__main__':
    upToFir()

Copy the code

After the above configuration, when we submit the code to the Git repository, we are ready to build. We need to configure the trigger check interval of Scan multi-branch pipeline. As long as the code is updated, the APK will be triggered and uploaded to fir.im. We can also see the execution time of related steps of the assembly line in BlueOcean, as shown in the figure

Modify the local.properties file according to config.yaml

Our project will put some of the configuration in the local local.properties, such as the module that the project is going to compile, the channel, the switch to compile the specific module, etc. In order to be able to modify the configuration file on each branch, we first need to copy the local.properties file from the local project to the root of the Jenkins server workSpace project. Then read the config.yaml parameter from JenkinsFile and write the corresponding key to local.properties,

Here’s how to implement this process:

1. First we will use Pipeline built-in readFile and writeFile

The idea is to get the contents of the original file through the readFile method and return a string object. And then I slice up the string object according to the newline, and I get a list object, and I go through the list, and I say if, and I find this line based on the Key, and I rewrite this line. Since we’re just rewriting in memory, we need to define a list object in advance to add both the overwritten and the unoverwritten items to the new list, and then define an empty string to iterate over the new list, adding newlines each time we concatenate a list element. This gives the string a complete content, which is then written back to the original config file using the writeFile method. The editfile. groovy code for this is as follows:

import hudson.model.*;

def setKeyValue(key, value, file_path) {
    // read file, get string object
    file_content_old = readFile file_path
    println file_content_old
    // Iterate over each line, determine, and replace the string
    lines = file_content_old.tokenize("\n")
    new_lines = []
    lines.each { line ->
        if(line.trim().startsWith(key)) {
            line = key + "=" + value
            new_lines.add(line)
        }else {
            new_lines.add(line)
        }
    }
    // write into file
    file_content_new = ""
    new_lines.each{line ->
        file_content_new += line + "\n"
    }

    writeFile file: file_path, text: file_content_new, encoding: "UTF-8"
}
return this
Copy the code

2. Local. The properties configuration:

sdk.dir=C\:\\Users\\xinmo\\AppData\\Local\\Android\\Sdk
market=google
build.module = chk
build.environment=product
compileSensorsSdk = false
Copy the code

3. Config. Yaml configuration:

#google huawei
market: Google
#Debug Release
buildType: Debug
#product(formal environment) stage(gray environment) test(test environment)
build.environment: test
# Declare modules that need to be compiled during development
#hrxs legendnovel chk teseyanqing xiaoshuodaquan
# English (includes novelCat foxNovel) Indonesia Freenovel PopNovel
build.module: foxNovel
#Sdk
compileSensorsSdk: true

Copy the code

4.JenkinsFile:

def loadValuesYaml(x){ def valuesYaml = readYaml (file: 'config.yaml') return valuesYaml[x]; } pipeline {// Agent nodes are configured to execute Android builds, iOS builds only, and Go projects. // Agent {label 'Android'} agent {label 'Android'} Agent any options {// Time out, Options: Retry (3) Timeout (time: 1, unit: 'HOURS')} environment{// A set of global environment variable key value pairs used in stages BUILD_TYPE = loadValuesYaml('buildType') BUILD_ENVIRONMENT = loadValuesYaml('build.environment') BUILD_MODULE = LoadValuesYaml ('build.module') COMPILE_SENSORS_SDK = loadValuesYaml('compileSensorsSdk')} Stages {// Here we already have the default checkout code Stage ('readYaml'){steps{script{println MARKET println BUILD_TYPE}  } } stage('set local properties'){ steps{ script{ editFile = load env.WORKSPACE + "/editFile.groovy" config_file = env.WORKSPACE + "/local.properties" try{ editFile.setKeyValue("market", "${MARKET}", config_file) editFile.setKeyValue("build.module", "${BUILD_MODULE}", config_file) editFile.setKeyValue("build.environment", "${BUILD_ENVIRONMENT}", config_file) editFile.setKeyValue("compileSensorsSdk", "${COMPILE_SENSORS_SDK}", config_file) file_content = readFile config_file println file_content }catch (Exception e) { error("Error editFile :" + e) } } } } stage('Build master APK') { when { branch 'master' } steps { bat "./gradlew clean assemble${MARKET}${BUILD_TYPE}" } post { failure { echo "Build master APK Failure!" } success { echo "Build master APK Success!" } } } stage('Build dev APK') { when { branch 'dev-hcc' } steps { bat "./gradlew clean assemble${MARKET}${BUILD_TYPE}" } post { failure { echo "Build dev APK Failure!" } success { echo "Build dev APK Success!" }} stage('ArchiveAPK') {archiveArtifacts(artifacts: 'app/build/outputs/apk/**/*.apk', fingerprint: true, onlyIfSuccessful: true) } post { failure { echo "Archive Failure!" } success { echo "Archive Success!" }} stage('Report') {// Display submission information steps{echo getChangeString()}} stage('Publish'){// Publish fir.im steps{bat './gradlew apkToFir' } post { failure { echo "Publish Failure!" } success { echo "Publish Success!" Emailext Body: 'apK version updated ', Subject:' APK uploaded successfully ', to: '1375****[email protected]'}}}}} //report log @noncps def getChangeString() {MAX_MSG_LEN = 100 def changeString = "" echo "Gathering SCM Changes..." def changeLogSets = currentBuild.changeSets for (int i = 0; i < changeLogSets.size(); i++) { def entries = changeLogSets[i].items for (int j = 0; j < entries.length; j++) { def entry = entries[j] truncated_msg = entry.msg.take(MAX_MSG_LEN) changeString += "[${entry.author}] ${truncated_msg}\n" } } if (! changeString) { changeString = " - No Changes -" } return changeString }Copy the code

6. Summary

Jenkins multi-branch pipeLine automated build, the build process can be managed by config.yaml, through the pipeLine syntax, write jenkinsFile to set the project’s overall build steps, Jenkins read the YAML configuration file, The groovy script is called to modify the local file local.properties and execute the packaging process. Groovy calls the Python script to upload fir.im. Through these steps to achieve the automatic build of each branch apK uploaded to fir.im.