Before I wrote some statistics in the pressure test script QPS may cause errors in several cases, today to fill a hole, the rest of the several kinds of test.

• Antecedents to review

• Performance test error analysis text version – on
• Performance test error analysis text version – next
• Comparative Study of Performance Test Errors (I)
• Comparative Study of Performance Test Errors (II)
• Comparative Study of Performance Test Errors (III)

Script with [comparative study on the performance test error (2)] (https://mp.weixin.qq.com/s/8oq9rSyCgxAiQAYrhHUCkg) is the same, the reason is not here.

JsonPath

There is a way to parse jSON-formatted data that several testing frameworks have similar designs, and REST Assured that I know of uses this approach and can say that it works very well. I’ve also covered the JsonPath framework before and wrapped it in Groovy features. As evidenced by the text:

• JsonPath Practice (1)
• JsonPath Practice (Ii)
• JsonPath Practice (Iii)
• JsonPath Practice (4)
• JsonPath Practice (5)
• JsonPath Practice (6)
• JsonPath utility class encapsulation
• JsonPath utility class unit test
• JsonPath validates classes that are both Groovy overloaded operator practices
• JSON objects mark syntax validation classes

Here I use the JSON data from the official Demo in JsonPath practice (1). Script modification is as follows:

        @Override
        void run(a) {
            times.times {
                def start = Time.getTimeStamp()
                sleep(0.05 )
                def instance = JsonUtil.getInstance(json)
                instance.get("\$.store.book[0].author")
                def end = Time.getTimeStamp()
                excutetimes.getAndIncrement()
                costs.add(end - start)
            }
            countDownLatch.countDown()
        }
Copy the code

It still takes 20 threads and 50 executions. The expected QPS should be 400.

INFO->The current user: oker, working directory: / Users/oker/IdeaProjects funtester/system coding formats: utf-8, Mac OS X system version: 10.16
INFO->QPS calculated by average time :357.7625529936
INFO->Calculate QPS by total time :349.7726477789
INFO->Error: 2.23%

Process finished with exit code 0
Copy the code

The expected QPS for this time should be 800 after 40 threads and 50 executions:

INFO->The current user: oker, working directory: / Users/oker/IdeaProjects funtester/system coding formats: utf-8, Mac OS X system version: 10.16
INFO->QPS calculated by average time :719.392113664
INFO->Calculate QPS by total time :702.7406886859
INFO->Error: 2.31%

Process finished with exit code 0

Copy the code

The error remains stable, and the error of distance expectation is relatively stable. Let’s do 40 threads, 100 executions. The expectation is still 800QPS. The results are as follows:

INFO->The current user: oker, working directory: / Users/oker/IdeaProjects funtester/system coding formats: utf-8, Mac OS X system version: 10.16
INFO->QPS calculated by average time :737.4869211304
INFO->QPS is calculated by the total time :729.527630859
INFO->Error: 1.08%

Process finished with exit code 0

Copy the code

The error reduction, and the expected distance reduction, should be responsible for running faster and faster Java hot code.

regular

This one is easier. Regex is one of the most common ways to validate and extract data. Static String jsonstr = json.toString() static String jsonstr = json.toString() The method is modified as follows:

        @Override
        void run(a) {
            times.times {
                def start = Time.getTimeStamp()
                sleep(0.05)
                Regex.isRegex(jsonstr, /0-\d{3}-\d{5}-/)
                def end = Time.getTimeStamp()
                excutetimes.getAndIncrement()
                costs.add(end - start)
            }
            countDownLatch.countDown()
        }
Copy the code

Here is the result of 20 threads and 50 executions:

INFO->The current user: oker, working directory: / Users/oker/IdeaProjects funtester/system coding formats: utf-8, Mac OS X system version: 10.16
INFO->QPS calculated by average time :370.6930105833
INFO->Calculate QPS by total time :361.0108303249
INFO->Error: 2.61%

Process finished with exit code 0
Copy the code

The error is not much, but it is much closer to the expected 400 QPS than JsonPath. Let’s try 40 threads, 50 requests.

INFO - > the current user: oker, working directory: / Users/oker/IdeaProjects funtester/system coding format: utf-88Mac OS X version:10.16INFO-> Calculate QPS by average time:739.8638650488INFO-> Calculate QPS by total time:721.240533718INFO-> Error is:2.52%

Process finished with exit code 0
Copy the code

The result is very close to JsonPath, and the error is relatively stable. Compared with JsonPath, it is larger, and the distance expected QPS error is stable. Here are 40 threads with 100 executions:

INFO - > the current user: oker, working directory: / Users/oker/IdeaProjects funtester/system coding format: utf-88Mac OS X version:10.16INFO-> Calculate QPS by average time:750.0328139356INFO-> Calculate QPS by total time:741.1524921253INFO-> Error is:1.18%

Process finished with exit code 0

Copy the code

The error decreased significantly, and the distance expected QPS increased, which basically accords with the empirical value.

abnormal

In practice, there are few exceptions. Generally, exceptions are caused by HTTP protocol exceptions or service verification failures. With those two, we should be gathering clues and getting ready to troubleshoot.

The methods are as follows:

        @Override
        void run(a) {
            times.times {
                def start = Time.getTimeStamp()
                sleep(0.05)
                try {
                    if (getRandomInt(3) = =1) fail()
                } catch (e) {

                }
                def end = Time.getTimeStamp()
                excutetimes.getAndIncrement()
                costs.add(end - start)
            }
            countDownLatch.countDown()
        }
Copy the code

The old rule is to run 20 threads, 50 times, and the expected QPS is 200:

INFO - > the current user: oker, working directory: / Users/oker/IdeaProjects funtester/system coding format: utf-88Mac OS X version:10.16INFO-> Calculate QPS by average time:373.9016638624INFO-> Calculate QPS by total time:360.7503607504INFO-> Error is:3.52%

Process finished with exit code 0

Copy the code

The error is larger than the first two factors, but close to the expected value, indicating that exception handling is fast. Here are 40 threads with 50 executions:

INFO - > the current user: oker, working directory: / Users/oker/IdeaProjects funtester/system coding format: utf-88Mac OS X version:10.16INFO-> Calculate QPS by average time:753.1396509198INFO-> Calculate QPS by total time:733.137829912INFO-> Error is:2.66%

Process finished with exit code 0

Copy the code

The error has been reduced, and other indicators are consistent with the previous two. It is a common fault in these matters. Let’s try 40 threads and 100 executions. The expected error should be reduced and the QPS distance expected to be stable:

INFO - > the current user: oker, working directory: / Users/oker/IdeaProjects funtester/system coding format: utf-88Mac OS X version:10.16INFO-> Calculate QPS by average time:757.9130863168INFO-> Calculate QPS by total time:743.9092430723INFO-> Error is:1.85%

Process finished with exit code 0

Copy the code

As expected, ha ha!

It seems that the impact of exception handling on performance is relatively small, and you may encounter fewer exceptions in normal times. I was worried, but NOW I think I was.

• The series is finally over!!

---

FunTester.Tencent Cloud Author of the Year,Boss direct hire contract author.Official GDevOps media partner, non-famous test development.

---

• FunTester test framework architecture diagram
• FunTester test project architecture diagram
• Dom-based XML file parsing class
• How to learn Java basics
• Soft start of performance test
• Java grid output class
• Manual testing or automated testing?
• Bind mobile phone number performance test
• Java multithreaded programming applied in JMeter
• Don’t always bend over in humility
• Details in the test case