Due to the requirements of the company, thousands of use cases on the APP end should be implemented automatically. To support the very unstable test environment and free selection of various branch environments, each module/service can be freely combined like building blocks, apK can be directly specified/uploaded on the platform when using, and the running status visualization should be realized in the process of execution. The test report is complicated and simple, and there are Word standard reports. And to achieve multiple devices in parallel/support multi-task execution. There are also a number of requirements such as support for buried point automation assertions.
It took me about a week to figure out what the platform was all about. But when I actually started writing use cases, I found a fatal problem, a problem that ruined all my previous efforts:
The number of use cases is simply too large.
If it’s just core use cases, fine, dozens, a hundred, two hundred. It’s not too much of a maintenance problem.
But when I reached thousands of use cases, I suddenly realized that I had been sloppy.
A real quantitative change causes qualitative change:
How long will it take me to write all of the thousands of use case scripts? Months? A year? Given that we have a lot of iterations going online every day, all of these use cases have a lot of logic to change every day to maintain. In other words, I may not have written a tenth of the previous use cases before they are almost dead. At this rate, there will never be a time when all use cases will work exactly. Even later maintenance costs need to be calculated on a monthly basis, and worse, the entire huge script code, tens of thousands of lines, is almost completely useless.
In this case, ios and Android are still two sets of code scripts, and maintenance costs double. Given the current size of our company, this is impossible to do, especially in the current unstable background of UI automation. Even if my use case logic is very good and intelligent, I am afraid it is just a useless thing.
However, I still did not give up, looking at the ios automation guy is on the verge of running away, I also gave him a reassuring: let alone thousands of use cases, is tens of thousands of, 100,000, I also have a way to solve. It’s just that this way, I’m afraid it might stir up a thousand waves. Success or failure, I’m afraid the consequences are not what we can afford.
Why do you say that?
Because I’m working on an architecture that automates our thousands of use cases directly. Later mature, should no longer need me to maintain, of course, write these scripts for the first time do not need me to write, all automatic temporary generation, automatic maintenance update.
That sounds strong and scary. Yes, the length of the link, the depth of the hierarchy, and the amount of resources consumed by this architecture all warn me:
If you fail, you’d better resign, or you’ll be fired.
If we are lucky to succeed, then this will eliminate tens of thousands of UI automation test engineers, even if we do not promote, I am afraid that our automation students will be embarrassed.
However, I consider that the human mind, after all, is not wasted on moving bricks and reworking the work, your countless debugging scripts, positioning elements, find_element_by_id, in my opinion is a waste of human intelligence, human intelligence should only be on the design of the work. In the testing world that is writing test points and test cases, translating scripts and writing code to locate elements and maintain scripts is a repetitive task that will eventually be replaced by computers. Even if I don’t take this step, I still can’t stop the wheel of history.
I then began the design of this huge architecture, starting with an analysis of the background and pain points and conflicts of our current automated scripts.
Then think about what the costs are, why they are incurred, where they are wasted, and how to avoid, or shift, the costs, so that the new small costs replace the old huge duplicate costs.
Then the hierarchy of the architecture and the analysis of each module and technology, put forward the requirements. However, the application of many technologies, layer upon layer, if any link is not successful, will be wasted, resulting in total failure. Including several technique if you want to start from 0 to the need to make a long time, fortunately, I have all done it six years the technology of these key nodes, such as automatic maintenance elements positioning, caught automatic, fully automatic assertions, 11 pool matrix analytical method, keyword driven, closure dynamically generated temporary script, platform building, Python and Django, automatic monitoring, BASIC principles of UI automation, concurrent process threads, Page-object, collaborative development and use design, automatic element entry, intelligent matching, complex script componentification, entity resolution, Bionic testing, 11 black box use case design, a few algorithms available on LeetCode, Links to use the database server interface technology, shell commands, the adb fast operating encapsulation, intelligent asynchronous assertions, use case platform cracked and report generation, python self-healing algorithm, automatic alarm, and so on and so on (part of the technology to create, so in a lone and I at to do I’m afraid to waste a lot of time cost). At current levels, it should be wobbly enough to get through the entire design link.
Of course, there are also risk estimation, personnel risk, scheduling risk, cost risk, technical risk, final effect accuracy risk and so on. These all need to be considered in the feasibility assessment.
Finally, there are the advantages and future expectations of the overall architecture.
The expectations and advantages are nice:
1. Cost: With continuous improvement of accuracy, running-in. Thousands of use cases can be automatically maintained, with almost no cost in the cost of automation for new use cases and almost no cost in the cost of automated maintenance for old use cases. In fact, it can also be said that there is no actual script to save, let alone maintenance? Because every time, I directly generated the automatic code according to many Chinese use cases written by the test students. It doesn’t last forever. Whereas previous costs were linear to exponential, the cost of this architecture is constant and shrinking.
2 regression: this state can completely change the embarrassing status quo of UI automation, the previous complete regression needs a lot of people a lot of days, this time only needs a little time, and does not waste any manpower. Even a small requirements iteration can enjoy super comprehensive full regression testing. The number of bugs online will be almost drastically reduced!
3. Energy liberation: Functional test students can save the execution time of each regression to do more meaningful things, such as improving use cases and test points, free testing, learning new technologies, etc. Automated test students can also not debug hard to write the same script, and do not have to maintain dozens of late, a few days and have to re-maintain such a waste of energy, can save energy, to improve the framework to make it more intelligent and accurate.
4. Technology Ecology: Many students’ whimsical technology designs are often aborted due to lack of practical use. However, this architecture is very low-level and huge, and almost any new ideas can find a suitable place on it. Moreover, the feedback will be relatively large. For example, the popular AI neural network deep learning can greatly improve the accuracy. It will be the cornerstone of many students’ performance in promotion and salary increase.
5. Future mainstream: You can expect automation to become foolishly simple in the next few years. The scripts we write today, the simple techniques we use to locate elements, are likely to be diluted by some update to the open source framework or some new technology. And this architecture is likely to be one of the mainstreams of the future.
6 strong: The characteristics of this architecture is our cost will not be wasted, but will overlay, when its mature enough, or the introduction of ai, automatically when more and more mature, it will slowly don’t need our engineer’s intervention, will reduce the risk, because personnel do not need functional test students such as regression, so even if someone leave also won’t delay scheduling, For example, if I’m the lead developer, even if I’m gone, the architecture will evolve on its own without me having to update it or maintain it. As opposed to a lot of previous automation projects where once the main process leaves, there’s basically no one to maintain it.
7 Meet initial expectations: Support various test environments. Use case combinations of various modules/services. For each user. Multiplayer development updates. Various installation packages specified. Equipment can be added, deleted, changed and checked. HTTP (S) request header/body assertion.
8. Small project capacity: Considering the large amount of code ultimately supported and the large number of logs, the dynamic generation technology is adopted so that a computer can run easily even with tens of thousands of scripts
9 multiterminal reuse: this is important, because the whole architecture of the bottom is removable, free joining together other languages/frameworks/that is to say, ios and android is common in a set of circumstances, not as before to write scripts and maintenance of the two sets of code, this architecture can support everything, all the end of a set of architecture is completely no problem.
Resolving use case conflicts: Our previous background is that the most important use cases are the core use cases, which are characterized by stability, but are very important and few in number. The other use cases are less important, extremely numerous, and constantly changing. While core use cases are best validated by human eyes, other use cases are best automated.
In the case of the automation group, UI automation applies to scenarios where use cases are stable and few in number. So the conclusion is that automation can only implement core use cases, so only script core use cases.
This is the point of conflict and contradiction. The fundamental solution is this new architecture. All new architectures are automatically generated and executed without discrimination. Then each group can take what it needs.
In the end, I can’t tell you more due to space limitations, so HERE are two places to give you some space for reverie:
1: What are the maintenance costs of UI scripts?
There are two points: first, use case logic changes, such as the order of operations. This part of the cost can be avoided by the new architecture. Second, element positioning changes, such as ID name changes, which can be avoided by the new architecture.
2. How to parse Chinese natural language into script?
Try to understand why you can understand this sentence:
Enter a user name and click the button to log in
How on earth do you understand this passage? Why do you understand many of them in reverse order? Why do you know you’re clicking on the login button instead of clicking on the username field? In fact, because the human brain is so fast, you take in this sentence in a flash, you don’t know how you parse and understand step by step, try to slow down your mind. And then you’ll see what’s going on with my algorithm here in my new architecture. This is not quite the same as deep learning AI we already have in the industry, which is constant error correction, the former is bionics, and maybe even true AI. But in my new architecture, it’s a mix of complementary approaches, so it’s more reliable.
Finally, I hope you can like it and share it, because this article is likely to be a historic moment in the field of testing.