Abstract: The last article “[technology gas station] Baidu intelligent testing of the three stages”, introduced the baidu intelligent testing three stages, this article we put forward to mining scenes as the driving way to orderly promote the three stages of intelligent testing rhythm of large-scale landing.

Where does the scene come from is the first question we have to face, so it is from the nature of the test activity to find the answer. Testing activities can be roughly divided into five steps: test input, test execution, test analysis, test positioning and test evaluation. The objectives of the five-step test activities are different, so the overall test activities cannot be generally intelligent, which is easy to cause confusion of objectives and affect the implementation of the ground.

The idea of our development is: ** A test activity in a test of a service is a scene as a pilot, so as to form the promotion mode of point, line, surface and body. ** As shown below:

01 Intelligent exploration in test input stage

Test input: Identify test behaviors, data sets, and build real-world environments that cover more test scenarios and higher code coverage. This phase determines the upper limit of problem detection capability of test activities. In traditional testing methods, the selection of test inputs largely depends on experience and historical cases.

In the exploration of intelligent test, test input can be further improved and recall ability in test stage can be improved through intelligent abnormal single test case generation, massive query to find the most reasonable Query, page traversal recommended action set, fuzz function or interface input parameters, etc.

For example, in the scenario of test flow expansion and preliminary screening, the initial screening of flow can be achieved by expanding the coverage of flow and expanding the range of test input at the same time, eliminating invalid characteristic data and feature re-screening can be achieved by means of feature segmentation, coverage penetration and pile insertion. Based on the ideas of control flow diagram and data modeling, abnormal use cases are generated by mutation strategy for value type and path frequency. Improve test coverage while ensuring test input completeness and execution efficiency.

02 Intelligent exploration in test execution stage

Test execution stage: combining test input set and change details, the quality activity of effectively executing the execution at a small cost without reducing the ability to find problems is essentially an efficiency problem.

The traditional approach is: do not screen and optimize all determined test cases, do not simplify the full execution of the test process, often have high redundancy of use cases, long tail use cases grouped abnormal, and ultimately lead to low efficiency of test execution and high resource consumption;

The methods of intelligent testing are as follows: test set selection, similarity de-duplication, group balancing, resource scheduling to intelligently execute the test process, thus greatly improving the test execution efficiency and reducing the cost; In this phase, we through the static evaluation (static scan use cases), dynamic assessment (implementation cases and combined process and the result of state to judge the cases quality), mutation test (injection source anomaly operator, test case is valid), Flaky monitoring (check out the related to weed out the source code is not case) means to identify all the execution of cases, Intelligent cancellation, skipping, clipping, sorting, combination and other strategies are used to ensure that the use cases can be effectively executed in the shortest time.

03 Intelligent exploration in test and analysis stage

In the test and analysis stage, the results of test execution are analyzed and judged as a whole system to confirm whether there are problems with the results of test execution. The analysis and exposure of problems as accurately as possible is essentially a quality problem. The recall problems in the test input stage will be disclosed.

The traditional practice is for experienced test experts to develop criteria for whether there are problems with each test task, including but not limited to the existence of specific indicators; The size and contrast of specific indicators; Threshold setting, such as this approach is more dependent on people’s judgment ability, with the change of the system and environment lead to insufficient anti-interference ability.

Intelligent scenario is to make intelligent decision analysis based on the historical performance of test execution tasks and determine whether the test execution results have problems.

For example, in the data test, exponential smoothing calculation method of historical data can be used to calculate the reasonable fluctuation range of data size, line number and content, and then in the next test task execution, intelligent automatic judgment of the task result is wrong.

2. In the business indicator test scenario, the threshold fluctuation range of each indicator can also be used for reference to the online fluctuation range and the data of the historical test task, and the output reasonable fluctuation range can be automatically determined by intelligent calculation.

3. Analyze whether there are problems in the screenshots during the execution of front-end use cases through visual technology, without worrying about leakage caused by manual judgment;

4. Determine whether memory leaks exist by DTW curve fitting method instead of the traditional threshold method.

Intelligent exploration in 04 testing and positioning stage

Test positioning, mainly to quickly determine the cause of “failure” of test activities in order to quickly repair, is essentially an efficiency problem. In testing activities, execution failure is often encountered. Under normal circumstances, we often check tools, environment, code and other situations to determine the cause of activity failure, and finally decide whether to conduct corresponding behavior. In the exploration of intelligence, we hope to quickly find the cause of test task failure through intelligent positioning. The first step is to find tools/environment issues to determine whether to rebuild or heal; The second step is to find the failure caused by the code, so as to help the r&d to quickly locate the problem, quickly repair and improve efficiency.

In this process, we explore intelligent location of test failure based on decision tree, and realize automatic rebuild and self-healing of tool failure. In the field of monitoring, we use change wall and online system relationship knowledge graph to locate the scope of monitoring problems to assist R&D or operations to make quick loss stop actions.

Intelligent exploration in 05 test and analysis stage

Test evaluation is the most neglected phase of test activities. Practitioners often judge whether a test is passed by red/green light because of a test task. While test evaluation is a combination of test execution and system change to assess the overall system risk of test activities to be implemented or implemented, so as to further guide us whether to test activities. In essence, it is a quality issue.

Test evaluation is first used to form a risk judgment paradigm from historical data, which is then used to assess the risk of new projects. At the stage of forming the paradigm, we explored the quality model based on testing the whole process data. We need to select appropriate risk factors, establish the system of data collection and change perception, collect data and transform it into features, and use machine learning to train a large amount of real data. Conduct qualitative (whether a factor is positive or negative to the risk) and quantitative (how much influence it has on the risk) analysis on the characteristics, and obtain the risk prediction model, which transforms the artificial experience into objective judgment. In the use stage of the project, since a data perception system has been established, new data can be directly input into the model to obtain immediate risk conclusions and the impact of features on risks, so that the quality can be seen. The accumulated project data can feed back the model and form an effective closed loop.

When the project is to be tested, we first conduct a test evaluation to determine whether the project needs testing and assist in judging how to test. After the appropriate quality activities have been taken, a risk assessment should be conducted to further verify the test results and identify risks. As a result, test decisions become more effective, cumulative, and sustainable.

———- END ———-

Baidu said Geek

Baidu official technology public number online!

Technical dry goods, industry information, online salon, industry conference

Recruitment information · Internal push information · technical books · Baidu surrounding