The business requirements
Assume that the leader of the company assigns you a performance test requirement as follows: 1: there are 1000 employees who log in to the platform to clock in during working hours, and they may log in and clock in several times. 2: Service peak hours are from 8:00 to 8:30, half an hour. 3:90% of users need to ensure that the response time is within 1s. It is guaranteed that 5000 punch cards can be completed within half an hour, and 90% of the service time is less than 1s. What is the maximum number of concurrent transactions in the system within half an hour? What is the maximum number of concurrent users per second?
Decomposition of demand
1:1000 registered users 2: the service period is half an hour, that is, 30 minutes. 3: Ensure that 90% of the service response time is less than 1s. 4: Test the maximum system concurrency within half an hour and the maximum number of concurrent users per second
Test model building & use case design
This requirement is typical for a scenario that counts concurrency based on load. First we need to design the business scenario
Model building
The selected test services are login service and attendance punch card service. The corresponding operation flow of the two services is as follows: 1. Login service – Operation flow: 1) Access the home page 2) Click “Please log in” to go to the login page 3) Enter the user name and password and click “Log In Now” to go to the home page 4) Click “Log Out” to go to the home page
2. Attendance Punch – Operation process: 1) Visit and open the home page 2) Click “Please log in” to jump to the login page 3) Input: user name + password, click “Log in now” to jump to the home page 4) Click “Attendance punch” to jump to the attendance page 5) Click “attendance” 6) click “Exit” to jump to the login page
Scene design
During performance testing, you should first design the test scenario and then design the script for the scenario. To truly reflect the performance of the object under test, you need to simulate the service scenario in which the object under test may have bottlenecks as much as possible. Then design test scenarios for the business.
Types of common test scenarios
There are several common performance testing scenarios: single service benchmark, single service stress test, single service load test, integrated service benchmark, integrated service stress test, and integrated service load test. There are four test types: benchmark test, stress test, and load test 1) Benchmark test Tests whether a specific service meets the performance indicators expected by the system design or users. If the login interface is expected to support 500 concurrent login users, the response time should not exceed the required value. If yes, the benchmark is considered complete; otherwise, it fails. During the benchmark test, the benchmark test is considered complete only when any of the performance indicators are successful. Benchmarks can be divided into concurrency benchmarks and traffic benchmarks, both of which are designed to verify that the desired goals are met.
2) The stress test tests the duration of continuous service of a specific business under the maximum load, so as to verify the stability of the business under test
The load designed during the stress test is based on the system benchmark test. For example, if the maximum number of concurrent logon interfaces is 500, the stress test load is set to 500. The ability of the server to continue serving under maximum system load is verified by changes in runtime duration.
3) Load test Test the maximum load that a specific business can bear and verify the maximum number of loads that the business under test can bear. For example, the maximum number of concurrent loads in the system benchmark test is 500. After repeated tests, the load is gradually increased and the optimal load of the business under test is finally obtained. Under optimal load, the system needs to meet various performance indicators. The following four scenarios are used to determine the performance test: ① Login and concurrent baseline ② Login and Service Volume baseline ③ Attendance and concurrent baseline ④ Attendance and service volume baseline
The number of service threads is determined
The number of threads needs to be set in the scenario design. This test is used as an example to describe how to support 5000 user logins within 30 minutes. The number of threads is calculated as follows:
Thread = BC*[t/(60* t)] t: time period to be investigated. In this example, t =30min t: time consumed by a single user to simulate the real user behavior as much as possible BC: Service volume: BC=5000 in this example, it takes time to test a single service using JMeter. 1) Log in to the service – Number of threads Determine the number of threads. Use the packet capture tool to write the script of “Log in and log out”. Collect the aggregated report data, add the transaction controller, summarize the overall transaction, and the response time is shown below
Based on the data in the preceding figure, if 90% sampling is adopted, the time for logging in to the system is 329ms. In actual cases, you also need to consider the thinking time for simulating the user to enter the user name and password, wait for the return to the home page after successful login, and wait for the return to the home page after exiting. Assume the following: User input user name and password: 5s If a timer is used to simulate the thinking time, the login and logout time of a single user is 0.329+5=5.329s
Calculate the maximum number of threads required for 5000 user logins within 30 minutes. Thread = 5000 x [5.329/(30 x 60)] = 14.80, rounded to 15
2) Check in – determine the number of threads using the packet capture tool, write the script of “login – check in”. Collect the data of the aggregate report, add the thing controller, sum up the whole thing, and the response time is shown as follows:
In the actual situation where 90% sampling is adopted and the time consumed for a single time is 1055ms after a user logs in, consider the thinking time for the following scenarios: 5s waiting time on the home page after the user enters the user name and password: 5s waiting time after the user logs in: 5s waiting time after the attendance is enabled: 3s use a timer to simulate the thinking time
Thread = 5000×[14.055/(30×60)] = 39.04, round 39
The explanation of the formula
Thread = BC*[t/(60* t)] : Thread = BC*[t/(60* t)] : Thread = BC*[t/(60* t)] It is the number of concurrent requests made by users or threads to the server per unit of time. For example, in the login business scenario above, the number is 15. Jmeter indicates that the system requires an average of 15 threads to initiate requests at the same time to support the traffic within a specified period. In a real user scenario, it indicates that a maximum of 15 users per second can initiate requests at the same time to support the traffic within a specified period
The average concurrent