This paper mainly introduces the performance test (performance optimization) completed in our company, which is divided into theory and practice. Previous reviews within the group were intended to make performance testing routine and standardized, but in reality, not much performance testing was done (even if some interfaces violated principle 258).
It has been a long time since I wrote this article. I would like to share some interesting things I have encountered recently.
Carnations for Women’s Day
Aerial view of the Tower of the China Water Conservancy Museum
Delicious beef and duck
Okay, let’s start the text.
Theory of article
Performance Test Purpose
Possible problems
- In the test or pilot trial run our system has been very well, but after the product is launched, in the actual environment, the user will always appear such as puzzling problem, such as system operation after a period of time slow, lead to users are not satisfied with our products, if you can avoid these problems, found that early?
- The system just went online, is in the trial run stage, how many users can support access?
- Our system has been running for a period of time. In order to ensure that the system can always provide users with a good experience during operation, what should we do?
- The number of users on this system will increase dramatically next year. Will our system still be able to support that many users?
- More and more people will use this system in the future. How many users can it support?
- As more and more information is stored, database queries become slower and slower.
Problems that can be solved
- Verify system stability
- When a new service is launched, verify that the new system can meet the system launch specifications
- Evaluate system performance
- Verify that there are architectural bottlenecks in the system
- Predict and plan the future capacity of the system
- Verify the effect of performance improvement, perform comparative analysis of performance tests, and provide basis and suggestions for performance tuning
The basic flow
Involve people
| The role | Role responsibilities |
|---|---|
| Software Test Engineer | Responsible for the preparation of the whole performance test plan and scheme, environment building, test cases, script writing, test data preparation, test implementation, test data analysis, test results acquisition, test report writing, to ensure the smooth completion of the performance test. |
| Business system development engineer | Provide performance test metrics, performance test impact points, assist in performance environment deployment, track, tune performance, or resolve program problems based on performance test results. |
| The product manager | Provide performance test indicators |
The test strategy
| strategy | concept | purpose | concerns |
|---|---|---|---|
| Capacity test | The load test verifies the limits that reflect certain application characteristics of the system, such as maximum user concurrency, maximum database records, etc. | 1. Obtain the performance data of the system at the inflection point; 2. Obtain the performance data of the performance degradation point. | 1. Set indicators for resource usage and other applications, such as CPU, memory, and disks. 2. The principle of 80/20 is generally adopted in the setting of indicators. |
| Tuning test | After establishing a performance baseline for the system or application, focus on whether performance is improved. By changing the configuration of the system, such as the DB connection pool, JVM configuration, or the number of concurrent threads handled in the application, changing the algorithm logic in the application, etc. | Whether system optimization improves performance. | Pay attention to whether the system performance can be improved after application or configuration changes. |
| The load test | Through simulation system of the concurrent users or carried by the request traffic load, in a way that is under constant pressure observation system performance under different pressure changes, such as response time, data throughput and system resource, to check the system, performance and features of the discovery system possible performance bottlenecks, memory leaks and other issues. | 1. Obtain the optimal QPS of the system and the resource utilization rate under the optimal QPS; 2. Determine whether the system can meet the performance requirements; 3. Get the baseline concurrency of the system. | Focus on performance variations and resource utilization under different pressures. |
| Pressure test | A load test, judge the system under strong pressure, whether there will be unrecoverable collapse phenomenon, such as stuck, abnormal phenomenon, also belongs to a destructive test. | 1. Get the QPS and resource utilization of the system breakdown point; 2. Obtain the number of concurrency in the system baseline. | Focus on non-functional defects that cannot be found by functional testing. |
Application Scenarios
| Applicable Service Scenarios | Adopt a testing strategy | Selects the purpose | A frequency | For example, |
|---|---|---|---|---|
| Performance tuning validation | Load & Configuration | Verify the performance optimization results and seek the balance point. | high | Employee roster – Import |
| Flash sale sale | Surge & high concurrency | Verify processing power under high concurrency | high | Koala 618 rush |
| Finding performance bottlenecks | Load & Capacity & Limit | Seek performance and comparison under different pressures | high | |
| Establish a performance baseline | Load & Capacity | Seek the performance inflection point of the system | In the | Employee roster – Import |
| Batch capability verification | Batch processing & High concurrency | Verify whether scheduled and batch services meet requirements | low | |
| High availability Dr Recovery | High concurrency & surge | This section describes how to verify the system recovery capability and the system survival time when exceptions occur | low | Kaola 618 promotion – Contact customer service, product reviews and other downgrade (limit stream 3W, business reviews do not show) |
| Verify the service before it goes online | Concurrency & Load & Capacity & Stability | Check performance and verify alarm thresholds | In the | Social Security system on line |
| System stability verification | Load & Stability | Verify the long-term stability of the system | In the | Million companies account |
| Short-term peak flow influx | Surge & high concurrency | For specific social media forum-type business needs | low | Press conference, live broadcast |
Access to accurate out
Performance test entry criteria
- Has produced the performance test indicators after the review, and has the impact points provided by the development;
- The server code version is stable, and there is no serious problem hindering the test after the development of internal test;
- Resources related to the test environment are in place, including software and hardware resources.
- Test plan written and passed the review;
- Test cases are complete and reviewed;
- Functional testing freezes.
The performance test passed the standard
- It is generally defined in terms of TPS, response time, or how much has been improved compared with the historical version/data, and user experience, so as to make the standard more scientific and closer to the actual business scenarios. If the business side gives a general standard, you can refer to the following points:
- Complete all tests according to the test plan.
- The tested product meets the expected performance indicators.
- There is no performance bottleneck of the tested product.
- If the above three points are not met, you can continue to refer to the following points:
- Part of the plan is not completed due to some reasons, and the reasons should be explained and confirmed by the three parties and unanimously approved.
- The product under test does not meet the expected performance index, but the performance index is within the acceptable range, and after the confirmation of three parties and unanimous approval, the end can be completed.
- The product under test has performance bottleneck, but the impact is small, there is no room for tuning for the time being, and it can be finished only after the confirmation of three parties and unanimous approval.
Matters needing attention
Construct a note for test data
- Simple scenarios, then complex scenarios, test various scenarios that may occur in the business.
- Performance testing, like functional testing, requires test cases that cover all possible scenarios for the business;
- Design use cases/scenarios from simple to complex, gradually covering more conditions; Both reflect different levels of performance and are indispensable;
- In a simple scenario, the “single point” performance of a function point, interface, or service may be measured in detail to identify performance bottlenecks and optimize code in a small range.
- After simple scenarios, more conditions will be added, or combined scenarios will be tested according to certain business processes and complex situations, to test the overall performance of the whole application or the whole system.
Considerations for performing performance tests
-
Close redis, Manggo, and other related caches to test the worst case.
- Cache is one of the methods of performance optimization, data, that isn’t frequent change in order to accelerate the access speed, improve the performance of reading, at the same time reduce the pressure of the database, the data can be in the cache, but the cache must be after the data is access to join will only take effect after the cache, at the same time, the cache is also need to refresh mechanism, used to update the data;
- In order to avoid the cache data invalidity or the cache does not take effect after the system goes online or restarts, the system access pressure and database pressure are high at this time. In the performance test, it is necessary to simulate the situation of no cache.
-
The script must be uploaded to the performance test platform after the JMeter script is uploaded to the performance test platform after the JMeter script is locally tuned.
-
Performance test platform execution pass rate is 100% (login interface → service interface), error rate is 0.00%;
-
Do it multiple times to minimize the chance of deviation.
Practical article
First understand the specific process of performance testing
How do I perform performance tests
Define performance test metrics
-
Performance test metrics must be clear. This is a metric that needs to be provided by the developer or the product. In the absence of clarity, discuss with them and finalize the final result.
-
Once the goal is clear, the performance test is half the battle! Otherwise, there’s no point in doing it.
Performance test review
-
Review performance test metrics in light of business scenarios.
-
Review performance test plans or use cases.
Application for Performance Environment
slightly
Performance Environment Deployment
- Configuring the performance environment
- Performance environment database
- Performance environment allocates machines
- Viewing Server Logs
scripting
Login Interface 1
Login Interface 2
Login Interface 3
Import (Import people – Click Import)
GetExcelPreview (import people – get tables)
ImportEmployee (Import person – Import form)
Script debugging
slightly
Script execution
Output performance test report
slightly
This is the end of today’s content, eat some fruit continue to watch “big River”. Strongly recommend “big River”, “Mountain and Sea feelings”, “Fu GUI”, tears eye conscience production.