I recommend reading iOS Unit Test Automation — Unit test Statistics and iOS Unit Test Automation — Code coverage Parsing.

All scripts covered in this article are hosted in the GitHub project UnitTestParser. Demo project: UnitTestDemo.

Introduction to the

Incremental code coverage refers to the code coverage of the new code, which is how much of the new code is covered by unit tests. Incremental code coverage can be calculated using the following formula

deltaCov = coveredDeltaLineCount / deltaLineCount
Copy the code

Where, deltaLineCount refers to the number of valid lines of code in the new code, and coveredDeltaLineCount refers to the number of valid lines of code in the new code covered by the unit test. Divide the latter by the former to get the coverage of the new code, or incremental code coverage.

Usage scenarios

Incremental code coverage ensures the level of unit testing for incremental code, which in turn ensures that the overall unit testing level continues to improve. In general, we count incremental code coverage in the following scenarios:

  • When a new PR or Merge Request (MR) is created, the system checks the incremental code coverage of the PR or MR and determines whether to Merge the PR or MR.
  • Detect incremental code coverage between releases and assess the level of incremental code unit testing for each release.
  • Monitor daily incremental code coverage regularly to ensure that the level of code unit testing continues to improve.

Incremental code coverage can well reflect the unit test level of newly added code. With the support of incremental code coverage data, we can control the overall unit test level of project code from a higher dimension and improve code coverage.

Incremental coverage calculation

Let’s go back to the formula for incremental code coverage:

deltaCov = coveredDeltaLineCount / deltaLineCount
Copy the code

We already know that deltaLineCount is the number of valid lines of code in the new code, and coveredDeltaLineCount is the number of valid lines of code in the new code that are covered by the unit test.

So if we want to calculate incremental code coverage, we need to do it in three steps:

  1. Find the lines of code that add files
  2. Get a code coverage report for an individual file, determining the coverage for each line
  3. Determines whether the new line of code is covered by unit tests

Below press these 3 steps to break each.

1. Added code statistics

How do I find the lines of code in which files have been modified by this commit /MR?

This won’t be a problem if you’ve read the article about getting Git delta code data. By parsing git diff data, you can retrieve the incremental code file and corresponding lines of code.

2. Single file coverage report

After obtaining all the new files through the new code statistics, we need to process each file individually to determine whether each line of the file is overwritten. Therefore, we must be able to get code coverage reports for individual files.

In the case of the UnitTestDemo project, you can use the following command to get code coverage for a single file from the.xcresult file, including the number of times each line of code was executed by unit tests.

# /path/to/file is the absolute address of the file when the unit test is executed
xcrun xccov view --archive --file /path/to/file /path/to/xcresult_file
Copy the code

The command output is as follows:

3:2:1: * * * 4: * 5: * 6: * 7:8: * * 9:10: * * 11: * 12, 13: * * 14:15: * * 16: * 17:18: * * 19:10 20:10 21: 10 [(24, 0, 1)] 22:1 23:9 [(1, 5, 1) (30, 0, 2)] 24:2 25:7 [(1, 5, 2)] 26: July 27:7 [(6, 0, 10)] 28:10 29: 31:10:30 * * 32:33: * * 34:35: * * 36: * 37:1 38:1 39:1 / (24, 0, 0) 40:0 41:1 [(1, 5, 0) (30, 0, 0)] 42: [(1, 5, 0)] 45:1 45:1 [(12, 0, 0)] 45:0 47:0 [(6, 0, 1)] 48:1 49:1 50: * 51: * 52: * 53:0 54: 55:0 0Copy the code

If an error message is displayed, run the xcrun xccov view –archive –file-list test.xcresult command to check whether the entered file path is correct.

The index on the left is the code line index of the file, and the number on the right represents the number of times that the line has been executed in the unit test, where * indicates that the line is not an executable code line, such as a blank line, etc. In the coverage calculation, these lines will not be considered as valid code lines and therefore will not be counted.

I’m sure this gives you an idea of what effective lines of code mean in the incremental coverage formula.

3. Determine whether the new line of code is overwritten

Through the above two steps, we have the files and lines of code for this modification, and whether each line is covered by unit tests. Then we just need to parse each file line by line. The statistical rules are as follows:

  • New executable code line: not followed by line index*
  • Overridden new lines of executable code: not followed by row index*And the number is greater than 0

The UnitTestParser project provides the deltacov.rb script to parse and output incremental code coverage.

For example:

╰─± Ruby deltacov. rb --xcresult-path=path/to/xcresult_file --proj-dir=./ --diff-file=path/to/ Diff_file --output-file=deltaCov.txtCopy the code

Output:

New code coverage: 1.0 New lines of executable code: 11 New lines of executable code to be covered: 11Copy the code

If an error message is displayed, run the xcrun xccov view –archive –file-list test.xcresult command to check whether the entered file path is correct.

The data is also written into the deltacov. TXT file for other tools to read.

╰─± cat deltacov.txt 1.0 11 11 11Copy the code

conclusion

The Scripts provided by the UnitTestParser project can quickly and accurately parse the number of incremental lines and the number of incremental code coverage lines to calculate incremental code coverage.