background

UI automated testing, that is, through the means of automation to control the machine simulation human manual operation. With the continuous development of GrowingIO business and the increasing number of new requirements, the task of regression testing is getting heavier and heavier. The existing testing resources are not enough to cope with the heavy regression testing tasks, and UI automation is urgently needed to replace manual regression testing, so as to free the manpower of regression testing to do more accurate tests. As a result, GrowingIO’s Web UI automation is introduced in the following two aspects:

1. Frame construction

2. Integrated quality platform

Frame structures,

PageObject

UI-automated testing, as we all know, is at the top of the test pyramid, with a low ROI. Its pain points are mainly reflected in:

1. Test cases are expensive to maintain, and minor changes in the way page elements are positioned or laid out may result in major changes in the previously written code;

2. Code redundancy, low reusability and poor readability.

In view of the above pain points and through a large number of investigations, I decided to use PageObject design mode, whose core ideas are six principles:

  • Public methods represent the services provided by the page
  • Don’t expose page details
  • Do not mix assertions with operational details
  • Method can Return to a new page
  • Do not put the entire page into PageObject
  • The same behavior produces different results and can encapsulate different results

Based on the above six principles and Growingio’s specific business situation, the directory level is designed as follows:

  • BasePage layer: A method that encapsulates some of the basic operations on a Web page, such as opening a browser, finding elements, taking screenshots, and so on
  • Component layer: Inheriting from the BasePage layer, it encapsulates methods to operate on common components in the page, such as the time Component
  • Page layer: This layer inherits the Component layer. Each method in this layer corresponds to a function of the current Page. Methods can call methods in the Component layer or methods encapsulated in the BasePage layer
  • TestCase layer: Invokes methods encapsulated in the business Page layer, writes business cases, and makes assertions

The actual project’s catalog is stratified as follows:

├ ─ ─ basepage │ └ ─ ─ base_page. Py ├ ─ ─ component │ └ ─ ─ element_design. Py ├ ─ ─ the conf │ ├ ─ ─ the conf. Py ├ ─ ─ datas ├ ─ ─ the log │ └ ─ ─ All the log ├ ─ ─ the py ├ ─ ─ page │ ├ ─ ─ home_page. Py │ ├ ─ ─ login_page. Py │ ├ ─ ─ main_page. Py │ ├ ─ ─ pytest. Ini ├ ─ ─ report │ ├ ─ ─ Requirements. TXT - run_all_cases.py ├─ testcase ├── conftest.py │ ├─ testcase.py │ ├── util ├─ util. Py

Selenium + Python

The language choice is Python, which is friendly to new people and familiar with the team members, so you can get started quickly.

At present, there are hundreds of Web UI automated test solutions on the market, which can be divided into the following categories based on the difference of the underlying technology:

1. WebDriver Protocol classes:

Selenium 3, WebDriverio, Protractor, NightwatchJS

2.Proxy JS Injection Class:

Selenium RC, TestCafe, Cypress

3. DevTool Protocol classes:

Such as the Puppeteer, Playwright t

We have chosen Selenium 3 for the following advantages:

  • Open source, free
  • Multi-browser support: Firefox, Chrome, Internet Explorer, Opera, Edge
  • Multi-platform support: Linux, Windows, Mac
  • Multi-language support: Java, Python, Ruby, C#, JavaScript, C++
  • Good support for Web pages
  • Simple (API: a method encapsulated in a class that exposes a usable interface to others), flexible (driven by a development language), and stable enough

The most important thing is that Selenium’s Grid solution, or distributed solution, is very mature. Distributed means that it consists of a Hub Node and several Node proxy nodes. Hub is used to manage the registration information and status information of each proxy node, and accept the request call from the remote client code, and then forward the requested command to the proxy node for execution, and finally summarize the execution results of each proxy node and return them to the remote client. Whether it’s integration with Jenkins or execution time requirements for use cases, distributed execution is the final state of UI automation, using docker-compose to create the Hub and Node nodes

The contents of the docker-compose.yml file are as follows:

version: '3' services: hub: container_name: selenium-hub image: selenium/hub restart: always ports: - 4445:4444 environment: HUB_HOST: hub health-timeout: 30 SE_NODE_SESSION_TIMEOUT: 30000 JAVA_OPTS: -Xmx1024m chrome: Image: Selenium /node-chrome-debug:3.141.59-20210311 container_name: Chrome_Test Restart: always depends_on: - hub ports: - 4446:5900 volumes: - /etc/hosts:/etc/hosts - /dev/shm:/dev/shm environment: JAVA_OPTS: -Xmx512m HUB_HOST: hub NODE_MAX_SESSION: 5 NODE_MAX_INSTANCES: 5 firefox: image: Selenium/Node-Firefox -Debug :3.141.59-20210311 container_name: Firefox_Test Restart: Always Ports: -4447 :5900 Volumes: Selenium/Node-Firefox -Debug :3.141.59-20210311 container_name: Firefox_Test Restart: Always Ports: -4447 :5900 Volumes: - /etc/hosts:/etc/hosts - /dev/shm:/dev/shm depends_on: - hub environment: - JAVA_OPTS=-Xmx512m - HUB_HOST=hub - NODE_MAX_SESSION=4 - NODE_MAX_INSTANCES=4

The GRID pattern performs a flow chart of use cases



 

Pytest

The PyTest framework is chosen as the framework for managing and organizing test cases, which has the following advantages:

  • Simple and flexible, easy to use, rich documentation
  • Parameterization is supported, allowing fine-grained control of the test cases to be tested
  • There are a number of third-party plugins with custom extensions, including allure-Pytest (perfect test reporting), PyTest-RerunFailures (repeated case failures), and PyTest-Xdist (multi-CPU distribution)
  • It works well with Jenkins

Fixtures are more flexible than traditional test frameworks (Setup/ tearDown). Fixtures are more flexible than traditional test frameworks (Setup/ tearDown).

  • There are independent names and they are activated by declaring their use from test functions, modules, classes, or the entire project
  • Implemented in a modular way, every Fixture can call each other
  • Fixtures are configurable and can be scoped to Function, Module, Class, or Session in the following order: Session > Module > Class > Function

The @PyTest.FixTrue decorator is used extensively in this project to decorate methods. The decorator name is passed into the test method as a parameter. It can be used to perform initialization prior to the test or to return to the database for test functions, especially with conftest files and yield

conftest.py

import pytest from selenium import webdriver from selenium.webdriver import DesiredCapabilities @pytest.fixture(scope='session') def init_driver(): if browser == "chrome": driver = webdriver.Chrome() elif browser == 'firefox': driver = webdriver.Firefox() elif browser == 'safari': driver = webdriver.Safari() elif browser == 'remote': capabilities = DesiredCapabilities.CHROME driver = webdriver.Remote(command_executor='http://localhost:4445/wd/hub', Desired_capabilities =capabilities) else: driver = "print(' Browser type not supported!! ') driver_obj = OpBasePage(driver) yield init_driver.open_op_url().login_op_by_gui(username, Driver_obj.close_browser ()

test_dashboard.py

class TestDataBoard:
    @pytest.fixture()
    def board(self, init_driver):
        yield init_driver.jump_to_board_by_url()

    def test_board_sort(self, board):
        board.click_button_go_to_board_manage().check_board_sort()

We can see from the above two files that the init_driver of the conftest.py file is passed in, and the board method in the test_dashboard.py file is decorated with the @pytest. fixTrue decorator. The test_board_sort test method is also passed in, so when the test method test_board_sort is run, the program is executed in the order

 

Allure

Allure is a lightweight and very flexible open source test reporting framework. It supports most testing frameworks, such as TestNG, Pytest, Juint, and so on. It is simple to use, easy to integrate with Jenkins, and shows trends in multiple test cases.

Allure Decorator:

Used in test cases

import allure import pytest @allure.feature("distribute-analysis") class TestDistributionAnalysis: @pytest.fixture() def distribution_analysis(self, init_driver): yield init_driver.jump_to_distribution_analysis_by_url() @allure.story("check distribute analysis") def test_analysis_success(self, distribution_analysis): with allure.step("create chart"): distribution_name, save_toast, distribute_detail_analysis = distribution_analysis.click_button_to_create_distribute_analysis().create_distribution_analysis() distribution_list_name, distribute_analysis = distribute_detail_analysis.click_crumb_to_distribution_analysis().get_first_distribution_chart_name() assert Distribution_name == Distribution_list_name, 'New distribution analysis single graph not shown on list page after creation' with allure.step("delete chart"): distribute_analysis.delete_first_distribution_chart()

Sample test report

Failed use case sample with embedded screenshot

At this point, the Web UI automation framework (PageObject + Selenium + Pytest + Allure) has been set up, and the overall execution process of the framework is as follows:

Integrated quality platform

The automation framework has been built, but this is only the first step. In order to facilitate the tracking and verification of the problems discovered by automation, the automation framework is integrated with the quality platform developed by ourselves, and it is connected with Feishu and JIRA to form a complete traceable closed-loop process. The specific process is as follows:

1. On the page of the quality platform, select the test environment address and project ID, and then click the “Start Web UI Test” button to execute the automation use case under the selected test environment and project

2. Upon completion of the execution of the automated use cases, the notification will be sent, and the data of each failed use case will be automatically crawled and displayed on the quality platform

3. Test personnel check, eliminate non-bug use cases, tick the remaining data, click “Submit Bug” button, that is, automatically create sub-bugs in batches on JIRA and assign them to the corresponding developers

4. When the developer has made the changes, repeat steps 1-3 until all the test cases pass

Flow chart after integration of quality platform

conclusion

This paper mainly introduces two parts of Web UI automation in GrowingIO framework building and integrated quality platform. The overall idea is: first, choose the appropriate framework and land, and second, the problems found by automation should be tracked and verified in time, so that the whole process forms a complete closed loop. Of course, the whole process of the Web UI automation building and integration quality platform mentioned above must still have a lot of need to be polished, I hope you give me advice.

 

​