SonarQube builds a notebook

The premise

This article documents the detailed process of building SonarQube services for PipleLine code scanning after cloud migration.

As I write this article (2020-05-17), the latest version of SonarQube is 8.3.1.

SonarQube profile

SonarQube is an open platform for code quality management that integrates thousands of automated static code analysis rules to improve code quality and security for developers, enabling them to write cleaner, more secure code. It provides three major functions:

• Code reliability support: Catch and alert errors in the code ahead of time to prevent undefined behavior from affecting end users.
• Application security support: Fix vulnerabilities that can harm applications and learn from security hotspotsAppSec(Easy to understand is to learn and identify new vulnerabilities).
• Technical debt support: Ensure that the managed codebase is clean and maintainable to improve developer productivity.

SonarQube currently supports 27 programming languages, basically covering projects written in the current mainstream programming languages:

The features mentioned above may be fairly general, but in fact, the development team can do the following with SonarQube:

• CI/CDThe process joins aSonarQubePart of the scan.
• Enforce code quality thresholds, which must be passed before moving on to the next process.
• If the code quality is below the threshold, the corresponding code should be adjusted in time.

The quality threshold can be customized. SonarQube has detailed panel information for each project, which provides the current health status of the project, classification and details of vulnerabilities at different levels, and multi-dimensional statistical information of vulnerabilities corresponding to submitters, so as to facilitate problem tracking and repair. For example, the author needed to run an assembly line when the project was launched in the last company, while SonarQube set different thresholds. For the old project, the lowest threshold was used: For some new projects, it is strictly required that the quality such as the severity of the error requirement is 0, etc. As long as the quality threshold fails to pass the check, the project cannot be online.

SonarQube installation

SonarQube (SonarQube, SonarQube, SonarQube, SonarQube, SonarQube, SonarQube, SonarQube, SonarQube, SonarQube, SonarQube, SonarQube, SonarQube) In Linux, you need to add a non-root user and modify a number of system parameters such as the maximum number of open files supported by the system. In addition, SonarQube is a Java application that requires a local installation of the JDK. SonarQube has dropped MySQL database support since version 7.9. In version 8.3.1, only memory mode, PostgreSQL, Microsoft SQL Server and Oracle storage engines are supported. In my research, PostgreSQL is relatively easy to install in all three databases in persistence mode. The installation of SonarQube services based on binaries is detailed below.

Software (System)	version
CentOS	7.x
OpenJDK	11.x
PostgreSQL	12.x
SonarQube	8.3.1

The author disabled the firewall on the tested VIRTUAL machine (WHOSE LAN IP address is 192.168.56.200). If the firewall is enabled, you need to open the corresponding port number.

Install JDK11

OpenJDK installation is relatively simple:

mdkir /data/openjdk
cd /data/openjdk
wget https://download.java.net/openjdk/jdk11/ri/openjdk-11+28_linux-x64_bin.tar.gz
#/data/ openJDK/jdK-11 by default
tar -zxvf openjdk-11+28_linux-x64_bin.tar.gz
Copy the code

If you don’t have a default JDK in your system, you can add it to /etc/profile:

vim /etc/profile

#Add the following to /etc/profile
export JAVA_HOME=/data/openjdk/jdk-11
export PATH=$JAVA_HOME/bin:$PATH

## Refresh environment variables
source /etc/profile
Copy the code

Test it out:

[root@localhost JDK -11]# Java -version openJDK version "11" 2018-09-25 OpenJDK 64-bit Server VM 18.9 (Build 11+28, Mixed mode)Copy the code

Install the PostgreSQL database

Installing PostgreSQL is also relatively simple, with the official documentation providing detailed steps:

yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm yum  install -y postgresql12 yum install -y postgresql12-server /usr/pgsql-12/bin/postgresql-12-setup initdb systemctl enable postgresql-12 systemctl start postgresql-12Copy the code

Modify the/var/lib/PGSQL / 12 / data/pg_hba conf configuration, open access to all the Host:

Then restart PostgreSQL:

systemctl restart postgresql-12
Copy the code

Switch the user to the PostgreSQL command line, and add a new database user sonar and a new database sonar standby:

su postgres
psql -U postgres
CREATE USER sonar WITH PASSWORD 'sonar';
CREATE DATABASE sonar WITH OWNER sonar ENCODING 'UTF8'
Copy the code

This has created a database with the name sonar, user name sonar, and password sonar.

Install SonarQube services

System parameters for ElasticSearch = System parameters for ElasticSearch = System parameters for ElasticSearch = System parameters for ElasticSearch = System parameters for ElasticSearch

## Maximum number of Vmas (virtual memory areas) that a process can own
sysctl vm.max_map_count
## Maximum number of open files at the same time
sysctl fs.file-max
#The maximum number of file descriptors that can be opened
ulimit -n
## maximum number of threads that can be started
ulimit -u
Copy the code

If the current session is user root, you can run the following command to modify the four parameters:

sysctl -w vm.max_map_count=262144
sysctl -w fs.file-max=65536
ulimit -n 65536
ulimit -u 4096
Copy the code

Otherwise you need to manually modify the/etc/security/limits file, add at the end of the file:

* soft nofile 65536
* hard nofile 65536
* soft nproc 4096
* hard nproc 4096
Copy the code

Add the following to the end of the /etc/sysctl.conf file:

vm.max_map_count=262144
fs.file-max=65536
Copy the code

The/etc/security/limits. Conf and/etc/sysctl. Conf must restart the server after the update.

Add user sonarqube (root cause ElasticSearch cannot start as root) :

adduser sonarqube
#This step requires the input of the password, which is also temporarily sonarqube
passwd sonarqube
#Assign permissions
chown -R sonarqube:sonarqube /data/sonarqube
Copy the code

Next download and install SonarQube:

mdkir /data/sonarqube cd /data/sonarqube wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-8.3.1.34397.zip uzip sonarqube - 8.3.1.34397. ZipCopy the code

Modify the configuration/data/sonarqube/sonarqube – 8.3.1.34397 / conf/sonar. The properties to add the following attributes:

sonar.jdbc.username=sonar
sonar.jdbc.password=sonar
sonar.jdbc.url=jdbc:postgresql://localhost:5432/sonar
Copy the code

After all the configuration is complete, you can try the console to start the SonarQube service:

CD/data/sonarqube/sonarqube - 8.3.1.34397 / bin/Linux - x86-64. / sonar. Sh the consoleCopy the code

If the startup is normal, the log is as follows:

Then Ctrl C exit the console and use./sonar.sh start to start the SonarQube service.

Here’s a list of possible problems:

• promptrootThe user could not start the problem becauseElasticSearchNot allowed to userootIf the user starts, create a common user.
• A message is displayed indicating that some folders do not have access permission, usually because newly created common users have not been assigned access permissionSonarQubeWrite permission of the directory.
• Indicates a file descriptor or thread limit, usually because no changes have been madevm.max_map_count,fs.file-max,ulimit -nandulimit -uParameter causes.

SonarQube use

SonarQube administrator’s initial account password is admin. If you need to change the password or assign users with different permissions, you can do so in the menu bar of the administrator.

Visit http://192.168.56.200:9000 to enter SonarQube WebUI, can go to market install a localization plugin Chinese Pack, after the completion of the installation, restart the service localization can be realized.

SonarQube provides different types of SonarScanner for code scanning and result submission, using Maven as an example. Maven settings. XML needs to introduce the following configuration (note that the parent tag already exists, adding the parent tag repeatedly will cause an exception) :

<settings>
    <pluginGroups>
        <pluginGroup>org.sonarsource.scanner.maven</pluginGroup>
    </pluginGroups>
    <profiles>
        <profile>
            <id>sonar</id>
            <activation>
                <activeByDefault>true</activeByDefault>
            </activation>
            <properties>
                <! -- Optional URL to server. Default value is http://localhost:9000 -->
                <sonar.host.url>
                  <! - this position needs to replace SonarQube service address, for example -- > http://192.168.56.200:9000
                  http://myserver:9000
                </sonar.host.url>
            </properties>
        </profile>
     </profiles>
</settings>
Copy the code

Http://sonar-maven-plugin.sonar-maven-plugin.sonar-maven-plugin.sonar-maven-plugin

<plugin>
    <groupId>org.sonarsource.scanner.maven</groupId>
    <artifactId>sonar-maven-plugin</artifactId>
    <version>3.7.0.1746</version>
</plugin>
Copy the code

After the configuration is complete, you only need to run the command in the project to scan. The first execution will be slow because you need to download a large number of rule libraries and plug-ins:

#Do not specify plug-in version execution
mvn clean install
mvn sonar:sonar

#Or specify the plug-in version to executeMVN org. Sonarsource. Scanner. Maven: sonar maven - plugin: ${plug-in version number} : sonarCopy the code

In addition, SonarQube parameters can be specified via in the POM file, such as:

<properties>
   <sonar.host.url>[...].</sonar.host.url>
   <ssonar.projectKey>[...].</sonar.projectKey>
   <sonar.projectName>[...].</sonar.projectName>
   <sonar.projectVersion>[...].</sonar.projectVersion>
   <sonar.login>[...].</sonar.login>
   <sonar.password>[...].</sonar.password>
   <sonar.sourceEncoding>[...].</sonar.sourceEncoding>.</properties>
Copy the code

Project scanning results submitted, can see list of items at http://192.168.56.200:9000/projects:

Click in to see the detailed report and statistics of the project after scanning:

summary

Quality management is an important part of DevOps. SonarQube is an excellent open platform for code quality management. The author migrated the service to a cloud before, and the pipeline configuration on the cloud can introduce SonarQube service in the form of plug-ins, which has the effect of low cost and high income.

References:

• SonarQube official documentation
• PostgreSQL – Linux downloads (Red Hat family)

Personal blog

• Throwable’s Blog

(C-1-D E-A-20200517)

Technical official account (Throwable Digest), push the author’s original technical articles from time to time (never plagiarize or reprint) :