Full code coverage can intuitively see the code coverage of the entire App, but it is often the incremental code coverage data that is useful or of concern. After continuous learning and exploration, we found that based on git diff capability, we obtained the incremental information we wanted through a series of processing, combined with the coverage information, and got an incremental coverage information.
Get the code diff
The project demo is used as an example
1. Find the double commit confirmation increment code
Git log to find the last (many) changes:
$ git logcommit a8692db2716eeb909aa9a9e48a89ac1a5368fa1f (HEAD -> master, origin/master, origin/HEAD) Author: denglibing5 <[email protected]> Date: Wed Jan 5 13:54:16 2022 + 0800 incremental code test commit 9 ad563ef1f3f1306aeec9c409d354cd695b1003b Author: denglibing5 <[email protected]> Date: Wed Jan 5 11:29:43 2022 +0800 updateCopy the codeA8692db2716eeb909aa9a9e48a89ac1a5368fa1f the modify the file is as follows:
2. Output incremental code information
Git diff: Git diff: git diff: Git diff: Git diff: Git diff
Select the first seven digits of commitid$ git diff 9ad563e a8692db diff --git a/Coverage_Gather.md b/Coverage_Gather.md new file mode 100644 index 0000000.. 2774839 - / dev/null. + + + b/Coverage_Gather md @ @ - 0, 0, + 1, 22 @ @ +Incremental code coverage+ + > total quantity of code coverage will be able to see the App intuitive code coverage, but often effective or are focused on the incremental code coverage data, here is based on the git ability, through a series of processing to obtain what we want incremental, combined with the coverage information, get a incremental coverage information. + +Diff = diff = diff+ + + + + + + + + + + + + + + + Reference [get git incremental code data] (HTTP: / / https://blog.jerrychu.top/2020/06/07/%E8%8E%B7%E5%8F%96git%E5%A2%9E%E9%87%8F%E4%BB%A3%E7%A0%81%E6%95%B0%E6 %8D%AE/) + diff --git a/Example/HDCoverageFramework/HDOCFramework.m b/Example/HDCoverageFramework/HDOCFramework.m index f617df7.. 5618bce 100644 --- a/Example/HDCoverageFramework/HDOCFramework.m +++ b/Example/HDCoverageFramework/HDOCFramework.m @@ -17 7 +17 14 @@ + (void)frameworkOCAction:(NSInteger)tag {NSLog(@"frameworkOCAction: %d", 2);
}
else if (tag == 3) {
+ NSLog(@"frameworkOCAction: new line");
NSLog(@"frameworkOCAction: %d", 3);
+ if (random() % 2 == 0) {
+ NSLog(@"frameworkOCAction: new line random() %% 2 = 0"); +},else {
+ NSLog(@"frameworkOCAction: new line random() %% 2 = 1");
+ }
}
NSDictionary * envir = [[NSProcessInfo processInfo] environment];
NSLog(@"envir : %@", envir); diff --git a/Example/HDCoverageFramework/HDSwiftFramework.swift b/Example/HDCoverageFramework/HDSwiftFramework.swift index 07d6661.. 20b6bb6 100644 --- a/Example/HDCoverageFramework/HDSwiftFramework.swift +++ B/Example/HDCoverageFramework/HDSwiftFramework. Swift @ @ - 28, 14, 28, 7 + @ @ public struct HDSwiftFramework {print("frameworkSwiftAction: 1")}else if (tag == 2) {
+ print("frameworkSwiftAction: new line")
print("frameworkSwiftAction: 2")
+ if (arc4random() % 2 == 0) {
+ print("frameworkSwiftAction: arc4random() %% 2 = 0") + +}else{+print("frameworkSwiftAction: arc4random() %% 2 = 1") +}}else if (tag == 3) {
print("frameworkSwiftAction: 3") diff --git a/README.md b/README.md index 97cdde6.. 824f767 100644 -- a/ readme.md +++ b/ readme.md @@-120,6 +120,7 @@class HDCoverageTools: {static var shared = HDCoverageTools(); __llvm_profile_write_file() + // func registerCoverage(moduleName: String) {let name = "\(moduleName).profraw"
print("registerCoverage, moduleName: \(moduleName)"@@-255,6 +255,16 @@$tree -l 3: 'tag == 3'"Framework(OC)-Case1/Case2"`. + + +# # summary+ + Full code coverage helps developers focus on logical flaws in changing code to better avoid online problems. Here is more about the 'Swift & Objective-C' project based on the full code coverage of the solution, there is no principle, only a simple process. Try several solutions along the way, and eventually rely on 'Cocoapods' to enable automated scripts. + + However, in real development, it is not possible to focus on full code coverage every time. The next article will cover incremental code coverage (./Coverage_Gather) + + +# # reference
[Source-based Code Coverage forStep by Step](https://nycode-jn.medium.com/source-based-code-coverage-for-swift-step-by-step-3df3c44c28d9) : Very detailed Swift code coverage tutorial, benefit a lot.Copy the codeHdocframework.m: – represents the deleted row, + represents the new row (modify is actually delete + new); @@ -17,7 +17,14 @@ indicates that 7 lines starting from line 17 have been deleted and 14 lines starting from line 17 have been added.
3. Incremental code data optimization
However, there are only 20, 22, 23, 24, 25, 26, 27 and 7 lines of code in hDOCFramework.m, as shown below:
Why doesn’t this match up with the sign at sign minus 17,7 plus 17,14, at sign at sign?
This is because Git diff does not compare row by row. Instead, it uses 3 rows as a comparison unit by default. As long as the data in the comparison unit has changed, it is considered that the unit has changed. So git diff shows that 14 lines have changed, even though only 7 lines have changed.
Incrementing 14 lines does not affect the presentation of diff (we can see that the red/green parts of the figure above are 4 lines), but it does confuse us in counting new/modified lines of code. For example, only 7 lines were modified this time, but Git Diff says that there are 14 lines changed, resulting in a large statistical data.
Git diff: Git diff: git diff
-U<n>
--unified=<n>
Generate diffs with <n> lines of context instead of the usual three. Implies --patch. Implies -p.
Copy the codeThat is, you can change the default behavior of Git diff by setting the value for Unified. Try executing again from the command line:
$ git diff 9ad563e a8692db --unified=0
diff --git a/Example/HDCoverageFramework/HDOCFramework.m b/Example/HDCoverageFramework/HDOCFramework.m
index f617df7..5618bce 100644
--- a/Example/HDCoverageFramework/HDOCFramework.m
+++ b/Example/HDCoverageFramework/HDOCFramework.m
@@ -19,0 +20 @@ + (void)frameworkOCAction:(NSInteger)tag {
+ NSLog(@"frameworkOCAction: new line"); @@ -20,0 +22,6 @@ + (void)frameworkOCAction:(NSInteger)tag {+if (random() % 2 == 0) {
+ NSLog(@"frameworkOCAction: new line random() %% 2 = 0"); +},else {
+ NSLog(@"frameworkOCAction: new line random() %% 2 = 1"); +}Copy the codeIt can be seen that the incremental data of this time is: @@ -19,0 +20 @@, @@ -20,0 +22,6 @@ and SourceTree visual match.
Parse incremental code data
Once you have the exact diff data, you need to convert the git diff output into something your code can read. For incremental code statistics, we want to get the following data: “The names of all change files, and all change lines for each change file.”
Git diff output contains both data, so we need to parse these data from the output. Use the existing script in the industry directly: diffParser.rb, copy the relevant Ruby files to the directory CoverageResult for easy management
The # diff file is stored in the CoverageResult directory
$ git diff 9ad563e a8692db --unified=0 > Example/CoverageResult/9ad563e.diff
$ cd├── Heavy Exercises ── Heavy Exercises ── Heavy Exercises ── Heavy exercises ── Heavy exercises ── Index.html │ └ ─ ─ style.css. CSS ├ ─ ─ GitDiff │ ├ ─ ─ trollop. Rb │ └ ─ ─ utils │ └ ─ ─ diffParser. Rb ├ ─ ─ MachOFiles │ └ ─ ─ │ ├── Bass Exercises - - Bass Exercises - │ ├── bass Exercises - - bass Exercises _CodeSignature ├ ─ ─ Profraw │ ├ ─ ─ HDCoverageDemo. Profdata │ └ ─ ─ HDCoverageDemo. Profraw └ ─ ─ hd_parse_profraw. Sh $ruby GitDiff/utils/diffParser.rb --diff-file=9ad563e.diff {"Coverage_Gather.md"= > [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22]."Example/HDCoverageFramework/HDOCFramework.m"=>[20, 22, 23, 24, 25, 26, 27]"Example/HDCoverageFramework/HDSwiftFramework.swift"=>[31, 33, 34, 35, 36, 37, 38]"README.md"=>[123, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267]}
Copy the codeMaybe we just need to focus on formatting file changes here. Therefore, I redeveloped the DiffParser. rb script and put it into the project.
In summary, the incremental code information of this time is as follows:
{"Example/HDCoverageFramework/HDOCFramework.m"=>[20, 22, 23, 24, 25, 26, 27]"Example/HDCoverageFramework/HDSwiftFramework.swift"=>[31, 33, 34, 35, 36, 37, 38]}
Copy the codeDifferent companies manage the old and new versions of the way in different ways, such as through tag, where the corresponding commitid can be found:
$git show - ref - 1951 tags d3e13100ce12b68d6b22f87cc5949bde4c09 refs/tags / 1.0.0 a90c8b98df705cc67eb19109c303826540f63c56 Refs/tags / 534 ed980c4f252042c931549b081e3938657f31b 1.0.1 refs/tags / 2 c7419fc3d5b957a3516797e0b0406e6c7ef0c4e 1.0.2 Refs/tags / 1.0.3 6630 e06f12e2de11075347be0c4beaabd6acbcb5 refs/tags / 4 c9cc93f49b390700fb3a1d8b4f3a3d6dd3ed70e 1.0.4 Refs/tags / 1.0.5 cd0a81b0e2978e0ddee87925cb283cf994731f53 refs/tags / 1.0.6 ba48e7df3e9430d9c10df590f40531608c9cbf9d Refs/tags / 1.0.7 be20cea26c4669e5d15498fed619f0019d2c5840 refs/tags / 1.0.8 ➜ CoverageResult git: (master) ✗ git show - ref 1951 d3e13100ce12b68d6b22f87cc5949bde4c09 1.0.0 refs/tags / 1.0.0 ➜ CoverageResult git: (master) ✗ git show - ref 1.0.1 A90c8b98df705cc67eb19109c303826540f63c56 refs/tags / 1.0.1Copy the codeClang-based incremental code coverage
As confirmed above for the incremental coverage solution, in using HDCoverage, the only products are *.profraw and *.profdata files. Git profData is a binary file, and even if you run the llvm-profdata command, you will not be able to associate the profdata with git delta code. Most of it is based on the product *.info under GCC to correlate incremental coverage data, which takes a lot of time.
During the process, I tried to modify HTML products with git incremental data, modify profData files and other methods, but failed to solve the problem. I also found a lot of information on the Internet, but no solution was found. After reading the llVM-cov and llVm-profdata-profile data tool documentation, I found the final solution:
1. Test incremental functionality to generate coverage data
The latest demo of the project is used as an example
Because this time commit to modify hDocFramework. m, hdSwiftFramework. swift so execute the test case, here I click respectively: After “Framework(OC)-Case2/Case3”, “FrameworkSwift) -case2 /Case3”, the App is backed to the background
Looking at the console, you can see the profraw file:
registerCoverage, moduleName: HDCoverageDemo HDCoverageGather filePath: /Users/denglibing/Library/Developer/CoreSimulator/Devices/5D01D4AA-40AE-4FC6-845C-391A94828EE3/data/Containers/Data/Appl ication/283906A5-1681-44A5-8522-126D29D2F148/Documents/HDCoverageDemo.profrawCopy the codeCopy hdCoverageDemo. profraw to CoverageResult/ profraw;
2. Generate code coverage files
Generate *. Profdata from the collected *. Profraw file using the llvm-cov command:
# xcrun llvm-profdata merge -sparse *.profraw -o *.profdata
$ xcrun llvm-profdata merge -sparse HDCoverageDemo.profraw -o HDCoverageDemo.profdata
Copy the code3. Reverse generate code coverage files under GCC
The *. Profdata file is converted into a *. Info file. Once you have *. Info, you can integrate it with git delta data.
# xcrun llvm-cov export macho-path -use-color -instr-profile=*.profdata -format=lcov > *.info
$ xcrun llvm-cov export /Users/denglibing/HDProject/HarryProject/iOS/HDCoverageDemo/CoverageResult/MachOFiles/HDCoverageDemo.app/HDCoverageDemo -instr-profile=HDCoverageDemo.profdata -format=lcov > HDCoverageDemo.info
Copy the code4. Git delta and code coverage integration
*.info is a readable file. Open hdCoverageDemo. info to see the coverage of this test (only hDocFramework. m and hdSwiftFramework. swift are displayed).
SF:/Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/HDCoverageFramework/HDOCFramework.m FN:12,HDOCFramework.m:+[HDOCFramework frameworkOCAction:] FNDA:2,HDOCFramework.m:+[HDOCFramework frameworkOCAction:] FNF:1 FNH:1 DA:12,2 DA:13,2 DA:14,0 DA:15,0 DA:16,2 DA:17,1 DA:18,1 DA:19,1 DA:20,1 DA:21,1 DA:22,1 DA:23,0 DA:24,0 DA:25,1 DA:26,1 DA:27,1 DA:28,1 DA:29,2 DA:30,2 DA:31,2 DA:33,2 LF:22 LH:18 end_of_record SF:/Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/HDCoverageFramework/HDSwiftFramework.swift FN:11,$s19HDCoverageFramework07HDSwiftB0V11saveProfileyyFZ
FN:26,$s19HDCoverageFramework07HDSwiftB0V20frameworkSwiftActionyySiFZ
FNDA:0,$s19HDCoverageFramework07HDSwiftB0V11saveProfileyyFZ
FNDA:2,$s19HDCoverageFramework07HDSwiftB0V20frameworkSwiftActionyySiFZFNF:2 FNH:1 DA:11,0 DA:12,0 DA:13,0 DA:14,0 DA:15,0 DA:16,0 DA:17,0 DA:18,0 DA:19,0 DA:20,0 DA:21,0 DA:22,0 DA:23,0 DA:24,0 DA:26,2 DA:27,2 DA:28,0 DA:29,2 DA:30,2 DA:31,1 DA:32,1 DA:33, 0 DA:35,1 DA:36,1 DA:37,1 DA:38,1 DA:39,2 DA:40,2 DA:41,1 DA:42,2 DA:43,2 DA:45,2 DA:46,2 DA:47,2 LF:36 LH:20 end_of_recordCopy the codeThe following are the keywords of the info file:
SF: <absolute path to the source file> The absolute path to the file
FN: <line number of function start>,<function name> The number of lines to start the method
FNDA:<execution count>,<function name> # Number of times this method is executed
FNF:<number of functions found> # Number of times the method was discovered
FNH:<number of function hit> # number of hits for this method (confused)
DA:<line number>,<execution count>[,<checksum>] # Number of lines of code, the number of times that line of code was executed
LH:<number of lines with a non-zero execution count> # Total lines of executable code, denominator of line of code coverage
LF:<number of instrumented lines> # Total lines of code executed in the numerator of line of code coverageSource: author: yuec links: https://juejin.cn/post/6844903904111493127 re the nuggets copyright owned by the author. Commercial reprint please contact the author for authorization, non-commercial reprint please indicate the source.Copy the codeGit delta: git delta: git delta: git delta: git delta
{"Example/HDCoverageFramework/HDOCFramework.m"=>[20, 22, 23, 24, 25, 26, 27]"Example/HDCoverageFramework/HDSwiftFramework.swift"=>[31, 33, 34, 35, 36, 37, 38]}
Copy the codeSF:/Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/HDCoverageFramework/HDOCFramework.m FN:12,HDOCFramework.m:+[HDOCFramework frameworkOCAction:] FNDA:2,HDOCFramework.m:+[HDOCFramework frameworkOCAction:] FNF:1 FNH:1 DA:20,1 DA:22,1 DA:23,0 DA:24,0 DA:25,1 DA:26,1 DA:27,1 LF:22 LH:18 end_of_record SF:/Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/HDCoverageFramework/HDSwiftFramework.swift FN:11,$s19HDCoverageFramework07HDSwiftB0V11saveProfileyyFZ FN:26,$s19HDCoverageFramework07HDSwiftB0V20frameworkSwiftActionyySiFZ FNDA:0,$s19HDCoverageFramework07HDSwiftB0V11saveProfileyyFZ FNH FNDA: 2, $s19HDCoverageFramework07HDSwiftB0V20frameworkSwiftActionyySiFZ FNF: 2:1 DA: 31, 1 DA: 33, 1 DA: 34, 0 DA: 35, 1 DA:36,1 DA:37,1 DA:38,1 LF:36 LH:20 end_of_recordCopy the codeAnd save it as HDCoverageDemo_gather. Info
5. Generate visual incremental code coverage files
$ genhtml -o html HDCoverageDemo_gather.info
Reading data file HDCoverageDemo_gather.info
Found 2 entries.
Found common filename prefix "/Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example"Writing .css and .png files. Generating output. Processing file HDCoverageFramework/HDSwiftFramework.swift Processing file HDCoverageFramework/HDOCFramework.m Writing directory view page. Overall coverage rate: lines...... : 78.6% (11 of 14 lines)functions. : 66.7% (2 of 3)functionsTree - L) $2. ├ ─ ─ HDCoverageDemo. Info ├ ─ ─ HDCoverageDemo. Profdata ├ ─ ─ HDCoverageDemo. Profraw ├ ─ ─ Info ├─ HTML ├─ Heavy Guitar ├─ emerald. PNG ├─ Heavy Guitar ├─ Heavy Guitar ├─ ├─ exclude.png ├─ exclude.png 2 directories, 12 filesCopy the codeOpen HTML /index.html to get the incremental code coverage page:
Click on a class in the Filename field to view specific information:
Combined with the previous code submission and the above visualization, this code change totaled 7 lines. After the test, 6 lines were executed, with line code coverage of 85.7%.
Above is the practice of total incremental code coverage.
Incremental code coverage scripting automation
The above two have verified the feasibility, but the intermediate process is not only many, and complex, easy to make mistakes. So it’s all wrapped up in a script that automates as many repetitive error-prone areas as possible.
1. Collect and obtain coverage files
Profraw = profraw; profraw = profraw = profraw; profraw = profraw;
2. One-click generation of code incremental coverage visualization
After the App runs, the CoverageResult directory will be automatically generated in the root directory of the project. The specific contents are as follows:
$Tree -L 2. ├──Git diff│ ├── Bass exercises. ├── bass Exercises# Git diff available data├ ─ ─ MachOFiles# Code generated binaries│ ├─ ├─ ├─ ├─ garbage ├─# Store code coverage files for execution├ ─ ─ hd_parse_profraw. ShScript to generate a full code coverage visualization page└ ─ ─ hd_parse_profraw_gather. ShGenerate a script for incremental code coverage visualization pages
6 directories, 3 files
Copy the codeCopy *. Profraw to CoverageResult/ profraw and execute:
A8692db: Code comparison commitid$ sh hd_parse_profraw_gather.sh 9ad563e a8692db Results: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/Results machOFiles: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/MachOFiles / Users/denglibing HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/Results exist disposeProfrawFiles profraws: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/Profraw disposeProfrawFiles profraw file: HDCoverageDemo.profraw =================================== findMachOFileName: HDCoverageDemo findMachOFilePath: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/MachOFiles/HDCoverageDemo.app/HDCoverageD emo =================================== disposeProfrawToHtmlByGenhtml, machoFileName: HDCoverageDemo machOFilePath: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/MachOFiles/HDCoverageDemo.app/HDCoverageD emo diff_file: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/Gitdiffs/9ad563e.diff coverage_info_file: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/Profraw/HDCoverageDemo.info ====== coverage_info_path: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/Profraw coverage_info_file: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/Profraw/HDCoverageDemo.info coverage_gather_file_path: /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/Profraw/HDCoverageDemo_gather.info ====== Reading data file /Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example/CoverageResult/Profraw/HDCoverageDemo_gather.info Found 2 entries. Found common filename prefix"/Users/denglibing/HDProject/HarryProject/iOS/hdcoverage/Example"Writing .css and .png files. Generating output. Processing file HDCoverageFramework/HDSwiftFramework.swift Processing file HDCoverageFramework/HDOCFramework.m Writing directory view page. Overall coverage rate: lines...... : 78.6% (11 of 14 lines)functions. : 66.7% (2 of 3)functions)
Copy the codeSee what is generated:
Tree - L $3. ├ ─ ─ GitdiffUtils │ ├ ─ ─ trollop. Rb │ └ ─ ─ utils │ └ ─ ─ diffParser. Rb ├ ─ ─ Gitdiffs │ ├ ─ ─ nine ad563e. Diff# 9AD563e compared to last time all change information│ └ ─ ─ nine ad563e. Json# 9ad563e Last processed change information (file name + changed line information)├ ─ ─ MachOFiles │ └ ─ ─ HDCoverageDemo. App │ ├ ─ ─ Base. Lproj │ ├ ─ ─ Frameworks │ ├ ─ ─ HDCoverageDemo │ ├ ─ ─ the Info. The plist │ ├ ─ ─ │ ├─ ├─ │ ├─ ├.info# Full code coverage data│ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ └ │ ├─ ├.info# 9AD563E incremental code coverage data compared to last time├ ─ ─ Results │ └ ─ ─ HDCoverageDemo_gather │ ├ ─ ─ HDCoverageFramework │ ├ ─ ─ amber. PNG │ ├ ─ ─ emerald. PNG │ ├ ─ ─ the gcov. CSS │ ├ ─ ─ Glass. PNG │ ├ ─ ─ the index - sort - f.h HTML │ ├ ─ ─ the index - sort - l.h HTML │ ├ ─ ─ index. The HTML │ ├ ─ ─ ruby. PNG │ ├ ─ ─ snow. PNG │ └ ─ ─ ├─ hd_parse_profraw.sh └─ hd_parse_profraw.sh 12 directories, 23 filesCopy the codeIncremental code coverage visualization
The visual file is in Results/HDCoverageDemo_gather, open index.html and you can see the output.
Of course, the development process in practice will be much more complex than this, such as different development environment for each company, multiple coverage data merge, multi-module multi-warehouse code incremental calculation, dynamic library special treatment, etc. Because the author searched for a lot of information and gained a lot before exploring the practice, but failed to find the direction and specific details that could cover the incremental coverage of Swift, this feasible practice came into being. It also fills a small gap in Swift development incremental code coverage. If you find a more efficient and reliable scheme, please inform us in time, and learn and build together.
Demo and script source address, welcome guidance +Star
Four, reference
Get Git delta code data: provides very detailed git delta code description and scripts, very nice