1. Classification of reporting framework
- XUnit Style XML report
- HTML report for the MVN Surefire plug-in
- Allure2 Multilingual test report
2. Generate test reports
2.1 xUnit Style XML report
All XUnit test reports are applicable, such as Java Junit, Python PyTest, etc
- 1) The following test classes are available
Junit4DemoTest
package junit4demo;
import org.junit.*;
import org.junit.runners.MethodSorters;
import static org.junit.Assert.assertTrue;
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class Junit4DemoTest {
@BeforeClass
public static void beforeAllTestCase(a){
System.out.println("I'm @beforeClass, I'm the first step");
}
@AfterClass
public static void afterAllTestCase(a){
System.out.println("I'm @afterClass. I'm the last step.");
}
@Before
public void beforeTestCase(a){
System.out.println("I'm @Before, come to me Before use case execution.");
}
@After
public void afterTestCase(a){
System.out.println("I'm @after, come to me After the use case executes.");
}
@Test
public void testDemoC(a){
System.out.println("testDemoC");
assertTrue(true);
}
@Test
public void testDemoA(a){
System.out.println("testDemoA");
assertTrue(false );
}
@Test
public void testDemoB(a){
System.out.println("testDemoB");
assertTrue(true); }}Copy the code
MVN test-dtest =Junit4DemoTest; Execution after we can target. The surefire – report. Surefile suite path TEST – junit4demo. See file Junit4DemoTest. XML, which is xUnit style XML report
testsuite
Display the name of the test class, the execution time, the number of cases executed, the number of errors, the number of skips and the number of failures:
property
Defines the execution environment, Java and Java library version, etc. :
testcase
Shows the name of the test class and the execution time of the test class
- If a use case fails, the failure information is printed under the Failure TAB, and the results of the run before the failure are printed:
- The full report is as follows:
<?xml version="1.0" encoding="UTF-8"? > <testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd" name="junit4demo.Junit4DemoTest" time="0.234" tests="3" errors="0" skipped="0" failures="1"> <properties> <property name="gopherProxySet" value="false"/> <property name="awt.toolkit" value="sun.lwawt.macosx.LWCToolkit"/> <property name="file.encoding.pkg" value="sun.io"/> <property name="java.specification.version" value="1.8"/> <property name="sun.cpu.isalist" value=""/> <property name="sun.jnu.encoding" value="UTF-8"/> <property name="java.class.path" value="/Users/qinzhen/Documents/TestDev/MyTraining/XUnit/target/test-classes:/Users/qinzhen/Documents/TestDev/MyTraining/XUnit / target/classes: / Users/qinzhen/m2 / repository/org/testng/testng / 6.14.3 / testng - 6.14.3. Jar: / Users/qinzhen/m2 / repository/c Om/beust jcommander / 1.72 / jcommander - 1.72. The jar: / Users/qinzhen/m2 / repository/org/apache - extras/beanshell/BSH/b6 / BSH - 2.0 2.0 B6. Jar: / Users/qinzhen/m2 / repository/org/junit/Jupiter/junit - Jupiter - engine / 5.5.2 / junit - Jupiter - engine - 5.5.2. The jar: / Users / Qinzhen/m2 / repository/org/API / 1.1.0 apiguardian apiguardian/apiguardian - - API - 1.1.0. Jar: / Users/qinzhen/m2 / repository/org Junit/platform/junit - platform - engine / 1.5.2 / junit - platform - engine - 1.5.2. Jar: / Users/qinzhen/m2 / repository/org/opentest4j / opentest4j / 1.2.0 / opentest4j - 1.2.0. Jar: / Users/qinzhen/m2 / repository/org/junit/platform/junit - platform - Commons / 1.5.2 / jun It - platform - the Commons - 1.5.2. Jar: / Users/qinzhen/m2 / repository/org/junit/Jupiter/junit - Jupiter - API / 5.5.2 / junit - Jupiter - API - 5.5.2. The jar: / Users/qinzhen/m2 / repository/org/junit/platform/junit - platform - runner / 1.5.1 / junit - platform - runner - 1.5.1. Jar: / The Users/qinzhen/m2 / repository/org/junit/platform/junit - platform - the launcher / 1.5.1 / junit - platform - the launcher - 1.5.1. Jar: / Users/q Inzhen/m2 / repository/org/junit/platform/junit - platform - suite - API / 1.5.1 / junit - platform - suite - API - 1.5.1. Jar: / Users/qinzhe N/m2 / repository/junit/junit / / junit 4.12-4.12. The jar: / Users/qinzhen/m2 / repository/org/hamcrest/hamcrest - core / 1.3 / hamcrest - c Ore - 1.3. The jar: / Users/qinzhen/m2 / repository/org/junit/Jupiter/junit - Jupiter / 5.5.0 - M1 / junit - Jupiter - 5.5.0 - M1. Jar: / Users/qin Zhen/m2 / repository/org/junit/Jupiter/junit - Jupiter - params / 5.5.0 - M1 / junit - Jupiter - params - 5.5.0 - M1. Jar: / Users/qinzhen/m2 / repository/org/apache/maven/surefire/surefire - testng - utils / 2.22.0 / surefire - testng - utils - 2.22.0. Jar: / Users/qinzhen/m2 / r Epository/org/apache/maven/surefire/surefire - grouper 2.22.0 / surefire - grouper - 2.22.0. Jar."/> <property name="java.vm.vendor" value="Oracle Corporation"/> <property name="sun.arch.data.model" value="64"/> <property name="java.vendor.url" value="http://java.oracle.com/"/> <property name="user.timezone" value=""/> <property name="java.vm.specification.version" value="1.8"/> <property name="os.name" value="Mac OS X"/> <property name="user.country" value="CN"/> <property name="sun.java.launcher" value="SUN_STANDARD"/> <property name="sun.boot.library.path" value="/ Library/Java/JavaVirtualMachines jdk1.8.0 _221. JDK/Contents/Home/jre/lib"/> <property name="sun.java.command" value="/Users/qinzhen/Documents/TestDev/MyTraining/XUnit/target/surefire/surefirebooter7582774321603719140.jar /Users/qinzhen/Documents/TestDev/MyTraining/XUnit/target/surefire 2019-10-15T08-12-59_318-jvmRun1 surefire5421147855410474454tmp surefire_03665361789485499624tmp"/> <property name="test" value="Junit4DemoTest"/> <property name="surefire.test.class.path" value="/Users/qinzhen/Documents/TestDev/MyTraining/XUnit/target/test-classes:/Users/qinzhen/Documents/TestDev/MyTraining/XUnit / target/classes: / Users/qinzhen/m2 / repository/org/testng/testng / 6.14.3 / testng - 6.14.3. Jar: / Users/qinzhen/m2 / repository/c Om/beust jcommander / 1.72 / jcommander - 1.72. The jar: / Users/qinzhen/m2 / repository/org/apache - extras/beanshell/BSH/b6 / BSH - 2.0 2.0 B6. Jar: / Users/qinzhen/m2 / repository/org/junit/Jupiter/junit - Jupiter - engine / 5.5.2 / junit - Jupiter - engine - 5.5.2. The jar: / Users / Qinzhen/m2 / repository/org/API / 1.1.0 apiguardian apiguardian/apiguardian - - API - 1.1.0. Jar: / Users/qinzhen/m2 / repository/org Junit/platform/junit - platform - engine / 1.5.2 / junit - platform - engine - 1.5.2. Jar: / Users/qinzhen/m2 / repository/org/opentest4j / opentest4j / 1.2.0 / opentest4j - 1.2.0. Jar: / Users/qinzhen/m2 / repository/org/junit/platform/junit - platform - Commons / 1.5.2 / jun It - platform - the Commons - 1.5.2. Jar: / Users/qinzhen/m2 / repository/org/junit/Jupiter/junit - Jupiter - API / 5.5.2 / junit - Jupiter - API - 5.5.2. The jar: / Users/qinzhen/m2 / repository/org/junit/platform/junit - platform - runner / 1.5.1 / junit - platform - runner - 1.5.1. Jar: / The Users/qinzhen/m2 / repository/org/junit/platform/junit - platform - the launcher / 1.5.1 / junit - platform - the launcher - 1.5.1. Jar: / Users/q Inzhen/m2 / repository/org/junit/platform/junit - platform - suite - API / 1.5.1 / junit - platform - suite - API - 1.5.1. Jar: / Users/qinzhe N/m2 / repository/junit/junit / / junit 4.12-4.12. The jar: / Users/qinzhen/m2 / repository/org/hamcrest/hamcrest - core / 1.3 / hamcrest - c Ore - 1.3. The jar: / Users/qinzhen/m2 / repository/org/junit/Jupiter/junit - Jupiter / 5.5.0 - M1 / junit - Jupiter - 5.5.0 - M1. Jar: / Users/qin Zhen/m2 / repository/org/junit/Jupiter/junit - Jupiter - params / 5.5.0 - M1 / junit - Jupiter - params - 5.5.0 - M1. Jar: / Users/qinzhen/m2 / repository/org/apache/maven/surefire/surefire - testng - utils / 2.22.0 / surefire - testng - utils - 2.22.0. Jar: / Users/qinzhen/m2 / r Epository/org/apache/maven/surefire/surefire - grouper 2.22.0 / surefire - grouper - 2.22.0. Jar."/> <property name="sun.cpu.endian" value="little"/> <property name="user.home" value="/Users/qinzhen"/> <property name="user.language" value="zh"/> <property name="java.specification.vendor" value="Oracle Corporation"/> <property name="java.home" value="/ Library/Java/JavaVirtualMachines jdk1.8.0 _221. JDK/Contents/Home/jre." "/> <property name="basedir" value="/Users/qinzhen/Documents/TestDev/MyTraining/XUnit"/> <property name="file.separator" value="/"/> <property name="line.separator" value="& # 10;"/> <property name="java.vm.specification.vendor" value="Oracle Corporation"/> <property name="java.specification.name" value="Java Platform API Specification"/> <property name="java.awt.graphicsenv" value="sun.awt.CGraphicsEnvironment"/> <property name="surefire.real.class.path" value="/Users/qinzhen/Documents/TestDev/MyTraining/XUnit/target/surefire/surefirebooter7582774321603719140.jar"/> <property name="sun.boot.class.path" value="/ Library/Java/JavaVirtualMachines jdk1.8.0 _221. JDK/Contents/Home/jre/lib/resources. The jar: / Library/Java/JavaVirtualMachine S/jdk1.8.0 _221. JDK/Contents/Home/jre/lib/rt. The jar: / Library/Java/JavaVirtualMachines jdk1.8.0 _221. JDK/Contents/Home/jre/lib / sunrsasign. Jar: / Library/Java/JavaVirtualMachines jdk1.8.0 _221. JDK/Contents/Home/jre/lib/jsse jar: / Library/Java/JavaVirt UalMachines/jdk1.8.0 _221. JDK/Contents/Home/jre/lib/jce jar: / Library/Java/JavaVirtualMachines jdk1.8.0 _221. JDK/Contents/H Ome/jre/lib/charsets. Jar: / Library/Java/JavaVirtualMachines jdk1.8.0 _221. JDK/Contents/Home/jre/lib/JFR jar: / Library/Java / JavaVirtualMachines/jdk1.8.0 _221. JDK/Contents/Home/jre/classes"/> <property name="sun.management.compiler" value="HotSpot 64-Bit Tiered Compilers"/> <property name="java.runtime.version" value="1.8.0 comes with _221 - bl1"/> <property name="user.name" value="qinzhen"/> <property name="path.separator" value=":"/> <property name="os.version" value="10.14.5"/> <property name="java.endorsed.dirs" value="/ Library/Java/JavaVirtualMachines jdk1.8.0 _221. JDK/Contents/Home/jre/lib/endorsed inside"/> <property name="java.runtime.name" value="Java(TM) SE Runtime Environment"/> <property name="file.encoding" value="UTF-8"/> <property name="java.vm.name" value="Java HotSpot(TM) 64-Bit Server VM"/> <property name="localRepository" value="/Users/qinzhen/.m2/repository"/> <property name="java.vendor.url.bug" value="http://bugreport.sun.com/bugreport/"/> <property name="java.io.tmpdir" value="/var/folders/55/lg_7qtv13d93hk9v7kjkn_p80000gn/T/"/> <property name="java.version" value="1.8.0 comes with _221"/> <property name="user.dir" value="/Users/qinzhen/Documents/TestDev/MyTraining/XUnit"/> <property name="os.arch" value="x86_64"/> <property name="java.vm.specification.name" value="Java Virtual Machine Specification"/> <property name="java.awt.printerjob" value="sun.lwawt.macosx.CPrinterJob"/> <property name="sun.os.patch.level" value="unknown"/> <property name="java.library.path" value="/Users/qinzhen/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/E xtensions:/usr/lib/java:."/> <property name="java.vm.info" value="mixed mode"/> <property name="java.vendor" value="Oracle Corporation"/> <property name="java.vm.version" value="25.221 - bl1"/> <property name="java.ext.dirs" value="/ Users/qinzhen/Library/Java/Extensions: / Library/Java/JavaVirtualMachines jdk1.8.0 _221. JDK/Contents/Home/jre/lib/ext: / Li brary/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java"/> <property name="sun.io.unicode.encoding" value="UnicodeBig"/> <property name="java.class.version" value="52.0"/> </properties> <testcase name="testDemoA on testDemoA(junit4demo.Junit4DemoTest)" classname="junit4demo.Junit4DemoTest" time="0.007"> <failure type="java.lang.AssertionError">java.lang.AssertionError at junit4demo.Junit4DemoTest.testDemoA(Junit4DemoTest.java:40) </failure> <system-out><! [CDATA[I'm @beforeClass, I'm @beforeClass, I'm @before, use case execution comes to me Before testDemoA I'm @after, use case execution comes to me After]]></system-out> </testcase> <testcase name="testDemoB on testDemoB(junit4demo.Junit4DemoTest)" classname="junit4demo.Junit4DemoTest" time="0.001"/> <testcase name="testDemoC on testDemoC(junit4demo.Junit4DemoTest)" classname="junit4demo.Junit4DemoTest" time="0"/> </testsuite> Copy the code
2.2 HTML report for the MVN Surefire plug-in
The MVN site generates the basic configuration of the project website and then executes MVN Surefire-Report: Report to generate the corresponding report:
2.3 Allure2 Test Report
2.3.1 What is Allure2?
- Take a look at allure2’s official explanation
It’s the next generation of Allure Report, a flexible, lightweight multilingual testing reporting tool that adds steps, attachments, metrics, and more
2.3.2 Installing Allure2
- For the installation of different systems, such as Linux, Mac and Windows, specific methods are given on the official website:
- Here to the Mac system demonstration, directly in the terminal to execute the command
brew install allure
After the implementationallure
The installation is successful if you see the following display, which shows the various command parameters of Allure:
Or use
allure --version
Command is also used to confirm the successful installation
2.3.3 Generation of allure2 simple report
- Look again at the official website to the tutorial:
- Perform the test in the IDE first, in
target/surefire-reports
Directory to generate the corresponding XML report- To perform
allure serve
The command
- The effect is shown in my
target/surefire-reports
There are two XML reports generated by test classes in the directory
after
allure serve /Users/qinzhen/Documents/TestDev/MyTraining/XUnit/target/surefire-reports
The following page is displayed: - The report analysis
- Overview: the home page
Overview
Is a general overview of which test classes are executed, and what are the results of pass and fail, etc - Categories: Categorize failed results
- Suites: Show specific implementation of all use cases
Graphs: Displays basic data in Graphs
- Timeline: a Timeline that can be pulled to review the test. (The official picture is quoted here. The demonstration project is not executed well and the effect is not obvious.)
- Behaviors: Test cases are grouped according to different characteristics
- Packages: Shows the Packages in which different use cases are located
- Overview: the home page
3, Allure detailed report generation
3.1 How Allure works
- Add the dependency and configuration of Allure to the test framework
- Perform the test
- Generate allure – results
- allure generate allure-results -o allure-report
3.2 generate allure – results
Allure is available in almost every language, and its website also provides a way to generate reports for various language frameworks, as shown below:
<properties>
<aspectj.version>1.8.10</aspectj.version>
</properties>
<dependencies>
<dependency>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-junit4</artifactId>
<version>LATEST_VERSION</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<testFailureIgnore>false</testFailureIgnore>
<argLine>-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.ja r"</argLine>
<properties>
<property>
<name>listener</name>
<value>io.qameta.allure.junit4.AllureJunit4</value>
</property>
</properties>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>${aspectj.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>.Copy the code
Note: the version of allure-junit4 relies on LATEST_VERSION can be viewed from the official website:
2) Execute the MVN Clean test again. After executing the MVN Clean test, you can see that there is a new folder allure- Results in the project
3.3 Annotation Features
3.3.1 Overview of annotations
The official website reads as follows:
annotations | instructions |
---|---|
@DisplayName | Set the use case presentation name |
@Description | describe |
@Step | Set the steps and complete the steps description |
@Attachments | Add attachments |
Links: @link, @issue, @tmslink | Link tests to certain resources, set bug numbers, and set use case numbers |
@Severity | Set the bug level to Blocker, Caitical, minor, Normal, trivial |
-
@Link, the website explains
From the instructions and examples provided on the official website, it can be seen that the Link address can be set through @link, but additional configuration is required in the first example
@Link
At present, my understanding is to show the address that needs to be linked, which has no other practical functions (there may be other functions, but the knowledge reserve is not enough, to be explored)@Link
Is to set the specific location and type of link,name
In is the configuration{}
{} can be removed from the configuration, so that name can be used as a single hyperlink name. It needs to be created in the same directory as testTest Resources Root
Folder of typeresources
And create a configuration fileallure.properties
allure.results.directory=target/allure-results allure.link.mylink.pattern=http://xxx.com/ allure.link.issue.pattern=https://example.org/issue/{} allure.link.tms.pattern=https://example.org/tms/{} Copy the code
== language is relatively obscure to describe, the following practical demonstration section will be much clearer ==
-
@Severity, the official website does not directly explain several levels of Severity. When we call this interface in IED, we can see the following levels through code prompts
3.3.2 Practical operation demonstration
1) Basic annotations with @Link
- Annotate the use case testDemoA
@DisplayName
、@Description
,@Link
,@Issue
,@Severity
- Create in the test sibling directory
Test Resources Root
Folder of typeresources
And create a configuration fileallure.properties
mvn clean test
->allure serve allure-results
Post-generated test report from the report can see the changes brought about by annotations:Clicking on bugAddress will take you directly to the project address I have set:
2) @step Step display
- new
login
Method and add@Step
Annotation, “login step”; In the method againtestDemoA
Call the method twice inlogin
:
Test results:
As you can see from the test results, the Allure report prints out the corresponding method call execution steps
3) @attachment screenshots
There are two approaches for screenshot functionality on the website, @Attachment annotation and Allure. Addattinfringement API, so here is API for demonstration
- Add a screenshot to the login() method:
Test results:
It can be seen from the test results that each time login is executed, screenshots are saved, so that listening can be set up during UI automation, screenshots are taken when use cases fail or screenshots are set up in key steps, and finally displayed in the test report for convenient analysis of test results
3.4 Dynamically generating test steps
You can dynamically add steps to use cases. In actual applications, you can insert log information for printing
3.5 Jenkins combination
Fill in the Jenkins section
4. Link address
Allure official GitHub: github.com/allure-fram… Github.com/allure-fram… Allure2 official GitHub: github.com/allure-fram… Qameta. IO /allure/#_ab…