Basic concepts of front-end automation testing

1. An overview of the

For those of you who have a little bit of development experience, there are always some lessons learned during the development process.

For example, when code complexity reaches a certain level and the number of maintainers is greater than one, you should be aware that you are becoming more cautious about developing new features or fixing bugs. Even if the code looks fine, there are still concerns about whether the Feature will introduce other bugs, or whether the fix will introduce other features.

When you want to refactor code in a project, you spend a lot of time doing regression testing.

All of these problems are caused by the basic manual testing method that most developers use, and the root cause is to introduce automated testing solutions.

In daily development, the completion of code is not the same as the completion of development, without testing, there is no guarantee that the code will work properly. Application testing is checking that the program is running correctly. Generally divided into manual testing and automated testing. Manual testing is not suitable for large projects, and it’s easy to forget to test a feature and spend most of your time doing regression testing.

An automated test is a test method that uses a computer program to check whether the software is working properly, that is, to check the code of the software under test with additional code. When the test code is written, it can be reused indefinitely. There are many ways to write an automated test script, including automatically executing the program through the browser, calling functions directly from the source code, and directly comparing the screenshots rendered by the program.

Automated testing can detect bugs and deficiencies early in the day, increase programmer confidence in robustness and stability, improve design, read feedback faster, reduce debugging time, and even facilitate refactoring.

Automated tests, of course, impossible to guarantee a program is completely correct, but in fact, in the actual development process, writing test automation code is usually the developer doesn’t like a link, in most cases, the front end after developers in the development of a function, just open the browser manually click view effect is correct, This code is rarely managed after that.

There are four kinds of common tests in front-end development. Unit test is to verify whether an independent unit works properly. Integration tests verify that multiple units work together. End-to-end testing verifies application interactions in a real browser environment in a machine-like manner from the user’s perspective. Snapshot tests validate UI changes to an application.

Unit testing

Unit testing is the process of running tests against the smallest part of a program. Normally the units tested are functions, but in front-end applications, components are also units under test. Unit tests can individually call functions in the source code and assert that they behave correctly.

/ / the source code
export default const sum(a, b) => {
    reurn a + b;
}

/ / test
import sum from './sum';

const testSum = () = > {
    if (sum(1.1)! = =2) {
        throw new Error('sum(1, 1) did not return 2');
    }
}

testSum();
Copy the code

Unlike end-to-end testing, unit tests are fast and take only a few seconds to run, so you can run them after every code change to get quick feedback on whether the change broke existing functionality. Unit testing should avoid dependency issues, such as no database access, no network access, and so on, and instead use tools to virtualize the runtime environment, which minimizes the cost of testing and requires less effort to set up the environment. However, because unit tests are independent, there is no guarantee that multiple units will run correctly together.

Common javascript unit testing frameworks include Jest, Mocha, Jasmine, Karma, Ava, and Tape.

Mocha and JEST are the two most popular unit testing frameworks at present. Basically, unit testing is between these two libraries. In general, JEST has complete functions and convenient configuration, while Mocha is free and flexible.

jest === mocha + chai + sinon + mockserver + istanbul
Copy the code

3. Integration test

Developers define integration test method is not the same, because it is for the front end, some people think that the browser running on the test environment for the integration test, some people think that has any test module dependency unit test is integration testing, also some people think that any rendering component tests are completely integrated test.

Integration testing is easier to obtain the correctness of software use process from the perspective of users. Integration testing is as opposed to writing documentation for the software. The lack of attention to the underlying code implementation details makes it easier to refactor quickly. Integration testing is faster to develop than unit testing.

Integration tests, however, cannot quickly locate problems when they fail, have low code coverage and are much slower than unit tests.

test('integration: new todo'.async() = > {const wrapper = mount(TodoApp);
    const todoInput = wrapper.find('[data-testid="todo-input"]');
    const text = 'Hello world';
    await todoInput.setValue(text);
    await todoInput.trigger('keyup.enter');
    const todoText = wrapper.find('[data-testid="todo-text"]');
    expect(todoText.text()).toBe(text);
})
Copy the code

4. End-to-end testing

End-to-end testing is the most intuitive and understandable type of testing. In a front-end application, end-to-end testing automatically checks whether the application is working from the user’s perspective through the browser.

Imagine, when writing a calculator application, and want to test two Numbers arithmetic method is correct, you can write an end-to-end test, open the browser, load the calculator application, click a button, click on the + button, click again on the 1 button, click the button =, 2 the result check screen display correctly.

describe('My First Test'.() = > {
    it('Visits the app root url'.() = > {
        cy.visit('http://localhost:8080/');
        cy.contains('h1'.'todos');
    })
    it('Visits the app root url'.() = > {
        cy.visit('http://localhost:8080/');
        const text = 'Hello World';
        cy.get('[data-testid="todo-input"]').type(`${text}{enter}`); cy.contains(text); })})Copy the code

End-to-end testing wasn’t fast enough, it took a few seconds to launch the browser, the website was slow to respond, it took 30 minutes to run through a set of end-to-end tests, and the test suite would take hours to run if the application relied entirely on end-to-end testing.

End-to-end debugging more difficult, to debug end-to-end test, need to open a browser and complete the user operation step by step in order to reproduce the bug, run the local debugging process is bad enough, if the test is on a continuous integration server failure rather than on the local computer, then the whole process can become worse.

Some of the most popular end-to-end testing frameworks are Cypress, Nightwatch, WebDriverio, and Ourselves.

5. Snapshot test

Snapshot testing, which takes a picture of a running application and compares it to a previously saved image and fails if the image is different, is helpful in ensuring that the application code changes and still renders correctly. A snapshot is a file that stores the page-generated structure and compares the new one with the old one.

6. Test the Pyramids

Which test type should you write with all these test types? It is written in general, and flexible distribution according to the situation.

If you really want to build automated tests for your software, one key concept you must know is the test pyramid. This means that tests need to be layered. The pyramid model is divided into unit test, integration test and UI test from bottom to top. The pyramid structure is because unit test has the lowest cost, while UI test has the highest cost. So unit tests write the most and UI tests write the least. It should also be noted that the higher the level of testing, the higher the pass rate will give developers confidence.

If you are developing pure libraries, it is recommended to write more unit tests and a few integration tests. If you are developing a component library, it is recommended to write more unit tests, snapshot tests for each component, and a few integration tests and end-to-end tests. If you are developing a business system, it is recommended to write more integration tests, unit tests for the library algorithms, and a small number of end-to-end tests.

7. Test coverage

Test coverage is an important indicator of the integrity of constant software testing. Mastering test coverage data is conducive to objectively understanding software quality, correctly understanding testing status and effectively improving testing work.

Test coverage can be measured by code coverage and requirements coverage.

The most famous test coverage is code coverage, which is a definition for software development and implementation. It focuses on what software code is being executed and what is not being executed during test case execution. The ratio of the number of code executed to the total number of code is code coverage.

According to the granularity of different code coverage can be further divided into source file coverage, class coverage, function coverage, branch coverage, statement coverage, etc. They are different in form, but the essence is the same.

Code coverage can generally be measured using the test coverage statistics that come with JEST. However, it is generally only suitable for white-box testing, especially unit testing, and for black-box testing, measuring coverage is much more difficult.

For black-box testing such as functional testing, integration testing, and system testing, test cases are usually based on software requirements rather than the software implementation involved. Therefore, a common measure of the completeness of such tests is requirements coverage, which is the ratio of the number of requirements covered by the test to the total number of requirements.

Depending on the granularity of requirements, the specific performance of requirements coverage is also different. For example, system testing is aimed at relatively coarse requirements, while functional testing is aimed at relatively fine requirements. Of course their essence is the same.

Measuring requirement coverage often doesn’t have a tool in place and requires manual calculations, particularly to mark the mapping between each test case and requirement. One of the common criticisms of code coverage is that 100% code coverage does not necessarily mean that the code is completely covered, because the order of execution of the code and the parameter values of the function can be extremely variable, and coverage of one condition does not mean coverage of all conditions.

For requirements coverage, 100% coverage is not everything, as requirements may have omissions or defects, and the mapping between test cases and requirements, especially whether the use cases actually cover the corresponding test requirements, may be questionable.

Code coverage and requirements coverage apply to different scenarios, have their advantages and disadvantages, and it is important to note that they are not mutually exclusive but complementary.

The most important thing about test coverage is to take the first step and consciously collect this data. Testing without coverage data is a bit like walking in the dark. With coverage and continuous testing, using and improving this data is a bright way to make testing better.

Since testing is so good, doesn’t all code need to be supported by test cases?

The test coverage should be combined with the test cost. For example, a public method that will not change often should try its best to make the test coverage close to 100%. For a complete project, it is suggested to cover 80% of the test cases in the shortest time in the early stage and gradually improve them in the later stage.

Active pages that change frequently do not need to approach 100% because of the high maintenance cost of constantly changing test cases.

In most cases, 100% code coverage doesn’t make sense as a goal, although it can be useful if it’s a critically important payment application that has a bug that could cost millions of dollars.

Not only is it time consuming to achieve 100% code coverage, but even when it does, testing doesn’t always find bugs and sometimes makes wrong assumptions. For example, if an API is called on the assumption that the API will not return an error, and the production environment is uncertain, it may return an error, and the program will crash as well.

8. TDD test driven development

Testing not only verifies the functionality of the software, but also ensures the quality of the code.

TDD stands for test-Dirven Development. Write test case code first, then write functional code for test case, so that it can pass, such an advantage is a good interpretation of the code and documentation, clear understanding of software requirements. It is a core practice and technique in agile development, as well as a software design methodology.

TDD is more about writing independent test cases, such as testing only a function point of a component, a tool function, etc.

Independent tests, different code tests should be independent of each other, one class for one test class, one function for one test function. Use cases should also be independent; each use case should not use the result data of other use cases, nor should the results depend on the order in which the use cases are executed. The development process involves a variety of tasks such as writing test cases, writing production code, refactoring code, etc. When doing different tasks, focus on the current role and don’t worry too much about other aspects of the details.

The test list, the code of function points may be many, and requirements may appear in succession, at any stage to add the relevant function points to the test list, then continue to work at hand, avoid omissions.

Test-driven, using test to drive development is the core of TDD. To realize a certain function and write a certain class or function, we should first write test code, make clear the use of this class or function and how to test, and then design and code it.

Write assertions first. When writing test code, you should first write assertions that judge the functionality of the code, and then write auxiliary statements as necessary.

Testability. Product code development should be as testable as possible, each code unit should be relatively simple in function, each class each function should do what it should do, do not get mixed up. Especially when adding new functions, do not add functions to the original code for the sake of convenience, for C++ programming, should consider the use of subclasses, inheritance, overloading and other OO methods.

Timely reconstruction, the structure of unreasonable and repetitive bad code, after the test, should be timely reconstruction.

Software development is very complex work, and taking small steps is a good way to reduce complexity.

TDD can ensure the code quality because the test is written first, so the possible problems are found in advance, and can promote the developer color examination, is conducive to the program module design, test coverage is also high, because it is written after the code, so the test cases are basically taken care of.

The disadvantages are also obvious. The amount of code can be quite large, with the test code being twice as much or more than the function code in most cases. Business coupling is high, test cases use some simulated data from business, and test cases need to be reorganized when business code changes. Concerns are also too independent, because unit tests focus only on the health of a single unit, making it impossible to guarantee the health of multiple units as a whole.

TDD is more suitable for developing pure libraries such as Lodash, React, and Vue.

9. BDD behavior drives development

The biggest problem with TDD is that developers can end up with something different from what they actually need, and BDD was invented to solve this problem.

BDD stands for Behavior Driven Development. By the system business experts, developers, testers work together to analyze the requirements of the software, and then write these requirements into a story, the developer is responsible for filling in the content, to ensure that the implementation effect of the program is consistent with the requirements.

Another key issue that BDD addresses is how to define the details of TDD or unit testing. Some common problems with bad unit testing are over-reliance on the implementation logic of the functionality being tested. This usually means that if you change the implementation logic, you usually need to update the test code even if the inputs and outputs change. This can make maintenance of test cases tedious and boring for developers.

BDD is an agile software development technology extended from test-driven development. The core of BDD is to solve the inconsistency between development and actual functional requirements under TDD mode. BDD eliminates the need to design tests for implementation details and instead focuses on behavior testing, which encourages collaboration between developers and non-developers from a product perspective. Since the core of BDD is focused on software functional testing, BDD is carried out more in combination with integration testing, which is a black box.

The development process is for developers and non-developers to discuss and validate requirements, establish requirements in an automated manner and verify consistency, finally implement the behavior described in each documented example, and start with automated tests to guide the development of the code.

The idea is to make each change small and quickly iterative, move it up every time more information is needed, add something valuable to the system every time you automate and implement a new example, and be ready to respond to feedback. The most popular ideal BDD solution is Cucumber, which uses gherkin syntax to transform functional requirements into requirements documents. Tests defined in descriptive natural language that customers, testers, and developers can understand and agree on are called gherkin syntax.

Scenario and feature are used to describe the scene, and steps are described by given, when and then.

Feature: Adding task Scenario: Enter the task name in the input box, press Enter to confirm, and output Given "Hello World" When in the input box, Then add a task Scenario named "Hello World" to the task column: Enter nothing in the input box, do not output Given "" When press enter in the input box Then add nothing to the task listCopy the code

Cucumber reads executable specifications in plain text described by Gherkin syntax and verifies that the software meets what those specifications say. The specifications contain multiple instances or scenarios.

Each solution is a list of Cucumber steps to perform, Cucumber verifies that the software meets the specification and generates a report indicating the success or failure of each case.

BDD focuses on product functionality and may not guarantee good code quality and test coverage.

BDD can be regarded as a bridge between requirements and TDD. It further scenarioizes requirements and more specifically describes what behaviors and scenarios the system should meet, making the input of TDD more elegant and reliable.

By focusing on the integrity of required functionality, BDD gives developers more confidence in the program. Testing only focuses on functions, not implementation details, which is conducive to decoupling the test code from the actual code. Since most of the tests are written for integration, it has better development efficiency compared with TDD.

However, BDD has low code coverage and is not as strict as TDD in ensuring code quality.

function	TDD	BDD
define	Test-driven development	Behavior-driven development
thought	From a code perspective, with the goal of producing high-quality code	From the user’s point of view, for the purpose of fulfilling functional requirements
The development process	Analyze requirements, write unit tests, run tests, write code, run tests, refactor and optimize, repeat the process	Developers communicate with product, test, customer and other personnel to determine functional requirements; Use unified format documentation to describe functional requirements; Establish test cases according to functional requirements; Run tests; Write code to implement functionality
Code coverage	high	general
Software security	general	high
Test type	Unit testing	Integration testing
Code decoupling	general	high
Development efficiency	general	high
The code quality	high	general
Test code volume	high	general

A comfortable development function library function using TDD scheme, development business system using BDD scheme.

10. Summary

When writing automated tests, keep in mind the purpose for which they are written. Often the purpose of testing is to save time, and if the ongoing project is stable and in long-term development, testing can be profitable. But tests should not be written if they take longer to write and maintain than they can save.

Of course, it’s hard to know how much time you can save by testing until you write the code, but you’ll find out over time. But if you’re prototyping on a short-term project, or iterating on an idea at a startup, you probably won’t benefit from writing tests.

There are two sides to every coin, and software testing is not a silver bullet. Although the benefits are obvious, not all projects are worth introducing a testing framework. After all, there are costs involved in maintaining test cases. For things that change frequently and have low reusability, such as active pages, it really doesn’t pay to have the development dedicated to writing test cases.

Scenarios suitable for testing are projects that require long-term maintenance and require testing to ensure that code is maintainable and functionality is not finalized. Writing test cases for stable projects or parts of projects is low maintenance. The parts that are reused many times, such as some common components and library functions, need to ensure quality because of multiple reuse.

Testing does provide considerable benefits, but not immediately. Just like buying insurance, it may not be used for more than ten years, unit test is the same, write a buy a rest assured, is a kind of guarantee for the code, there are bugs detected as soon as possible, no bug is the best, always can not say that write so many unit tests, the results can not detect the bug waste time.