The Hogwarts Testing Institute is a gold sponsor of python-UI Automator2.

I. Background introduction

Google officially provides an Android automation testing tool (Java library), based on the Accessibility service, with strong functionality, you can test third-party APPS, obtain any control properties of any App on the screen, and perform arbitrary operations on it. But there are two drawbacks:

1. Test scripts can only use the Java language;
2. The test script must be packaged as a JAR or APK package and uploaded to the device to run.

In practice, we want the test logic to be written in Python and to be able to control the phone while running on the computer. Therefore, based on this purpose, the python-UIAutomator2 open source testing tool was developed, which encapsulates Google’s own UIAutomator2 testing framework and can run on any system that supports Python. The current version is V2.10.2.

GitHub

https://github.com/openatx/uiautomator2
Copy the code

Two, the working principle

As shown in the figure, Python-UI Automator2 is mainly divided into two parts: Python client and mobile device

• Python: Run scripts and send HTTP requests to mobile devices.
• Mobile device: THE HTTP service encapsulated with UIAutomator2 is run on the mobile device, which parses the received request and converts it into THE code of UIAutomator2.

The whole process:

1. Install ATX-Agent (daemon process) on mobile device, then ATX-Agent starts UIAutomator2 service (default port 7912) to listen;
2. Write test scripts and execute them on a PC (equivalent to sending HTTP requests to the server side of a mobile device);
3. The mobile device receives the HTTP request from the PC through WIFI or USB and performs the specified operation.

Installation and startup

3.1 installation uiautomator2

Install using PIP

pip install -U uiautomator2
Copy the code

After the installation is complete, use the following Python code to check that the environment is configured successfully

Note: All the code in this article needs to import uiAutomator2 library. For simplicity I use U2 instead and D stands for driver

D = u2.connect() print(d.info)Copy the code

If the device information is correctly displayed, the installation is successful

Note: you need to install adb tools and configure the system environment variables to operate the phone.

Installation problems can be queried in the Issue list:

https://github.com/openatx/uiautomator2/wiki/Common-issues
Copy the code

3.2 installation weditor

Weditor is a browser-based UI viewer that helps us see where UI elements are located.

Since UIAutomator is an exclusive resource, uiAutomatorViewer is not available when ATX is running. This tool is needed to reduce frequent atX starts and stops

Install using PIP

pip install -U weditor
Copy the code

Check whether the installation is successful

weditor --help
Copy the code

The installation is successful if the following information is displayed

Run weditor

Python -m weditor# or run weditor directly from the command lineCopy the code

Fourth, element positioning

4.1 Usage

D (positioning mode = positioning value)# example: Element = d(text='Phone')# return a list of elements if no element is found. Click ()# print(element.count)Copy the code

4.2 Supported Location modes

Ui2 support in android UiSelector in the class all the way to locate, can detailed in this web site to see developer.android.com/reference/a…

The overall content is as follows, and all properties can be viewed through the Weditor

The name of the description

Texttext is the element with the specified text textContainstext is the element with the specified text textMatchestext is the element that matches the specified re textStartsWithtext Is the element that starts with the specified text classNameclassName Is to specify the name of the class element classNameMatchesclassName class names conform to the specified element of regular descriptiondescription descriptionContainsdescription is specified text elements Containing the specified text element in descriptionMatchesdescription conform to the specified element of regular descriptionStartsWithdescription begin with specified text by checking element, element checkable parameter is True, Falsechecked A selected element, usually used for check boxes, with an argument of True, FalselongClickable a clickable element with an argument of True, Falsescrollable Scrollable elements with True, Falseenabled Activated elements with True Falsefocusable elements with True Falsefocusable elements that should be focused Falseselected The currently selected element, which is True. FalsepackageNamepackageName element to specify the package name packageNameMatchespackageName resourceIdresourceId to conform to the regular elements For the resourceIdMatchesresourceId element at the specified content to conform to the specified regular elements

4.3 Localization of child elements and siblings

Child **** element location

child()

D (className=" android.widget.listview ").child(text="Bluetooth")# The following two ways are a little bit inaccurate, D (className=" android.widget.listview ")\. Child_by_text ("Bluetooth",allow_scroll_search=True)d(className="android.wid "is not recommended get.ListView").child_by_description("Bluetooth")Copy the code

Sibling element localization

sibling()

ImageView (text="Google").sibling(className=" Android.widget.imageview ")Copy the code

Chain calls

D (className = "android. The widget. The ListView", resourceId = "android: id/list") \. Child_by_text (" Wi ‑ Fi ", className="android.widget.LinearLayout") \ .child(className="android.widget.Switch") \ .click()Copy the code

4.4 Relative Positioning

Relative positioning is supported on the left, right, top, bottom, i.e. around an element

D (A). The left (B), choose # A Bd (A). On the left side of the right (B), # choose A to the right of the Bd (A). The up (B), D (text=' wi-fi '). Right (resourceId='android:id/widget_frame')Copy the code

4.5 Common APIS for elements

The table annotates class property methods decorated with @property, as shown in the example below

d(test="Settings").exists
Copy the code

methods

Description Returned Value Remarks

Exists () determines whether an element exists True,Flase@propertyinfo() returns all information about the element dictionary @propertyget_text() returns the element text string set_text(text) sets the element text None Clear_text () Clears element text None center() returns the element’s center point position (x,y) based on points across the screen

Exists Other usage methods:

= 'Wi ‑ Fi' d.e xists (text, timeout = 5)Copy the code

Info () Output information:

{ "bounds": { "bottom": 407, "left": 216, "right": 323, "top": 342 }, "childCount": 0, "className": "android.widget.TextView", "contentDescription": null, "packageName": "com.android.settings", "resourceName": "Android: ID /title", "text":" Wi-Fi ", "visibleBounds": {"bottom": 407, "left": 216, "right": 323, "top": 342 }, "checkable": false, "checked": false, "clickable": false, "enabled": true, "focusable": false, "focused": false, "longClickable": false, "scrollable": false, "selected": false}Copy the code

You can obtain all attributes of the element individually from the above information

XPATH 4.6 positioning

Because Java UIAutoamtor does not support xpath by default, this is an extension of UI2 and will be slower than other positioning methods

In xpath localization, the description localization in UI2 needs to be replaced with Content-desc and the resourceId needs to be replaced with resource-ID

Method of use

# returns only one element, if can't find the elements, will be submitted to the XPathElementNotFoundError mistake # if found more elements, Default returns 0 d.x path (' / / * [@ the resource - id = "com. Android. Launcher3: id/icon"] ') # if there are multiple return element, you need to use all # () method returns a list of all method to be used, when the element was not found, Not an error, returns an empty list d.x path (' / / * [@ the resource - id = "com. Android. Launcher3: id/icon"] "). The all ()Copy the code

5. Device interaction

5.1 click

D (text='Settings').click()# click until the element disappears, timeout 10, click interval 1d(text='Settings').click_gone(maxRetry =10, interval=1.0)Copy the code

5.2 long press

d(text='Settings').long_click()
Copy the code

5.3 drag

Drag cannot be used when Android<4.3

Drag Setting to Clock within 0.25s Drag_to (text="Settings"). Drag_to (text="Clock", Duration =0.25)# drag Settings to a point on screen d(text="Settings"). Drag_to (877,733, duration=0.25)# Drag between two points, Drag from point 1 to point 2d.drag(x1,y1,x2,y2)Copy the code

5.4 slide

There are two slides, one on the driver and one on the element

Operation on element

Slide from the center of an element to its edge

# Swipe up on Setings. Swipe ("up", steps=20). Swipe ("up", text="Settings").Copy the code

Driver operating on

That is to operate on the entire screen

D.wiper ()x1 = x / 2y1 = y * 0.1y2 = y * 0.9d.wiper ()x1 = x / 2y1 = y * 0.1y2 = y * 0.9d.wiper ()Copy the code

Driver slide extension method, can directly achieve sliding, do not need to encapsulate their own registration points

# support for sliding # "left", "right", "up", "down"# slide d.swipe_ext("down")Copy the code

5.5 Two-finger Operation

Android > 4.3

Operation on elements

d(text='Settings').gesture(start1,start2,end1,end2,)# = 'Settings' operation d (text), gesture ((525960), 613112 (1), (135622), 882154 (0))Copy the code

Encapsulated zoom in and out operation

# zoom in d(text="Settings").pinch_out() # zoom in D (text="Settings").pinch_out()Copy the code

5.6 Waiting for elements to appear or disappear

D (text="Settings"). Wait (timeout=3.0) Return True False,timout defaults to wait time set globally d(text='Settings').wait_gone(timeout=20)Copy the code

5.7 Scroll Page

Set the scrollable property to True;

Type of roll: Horiz horizontal, vert vertical;

Rolling direction:

• Forward forward
• Backward backward
• Roll toBeginning
• ToEnd scrolls to the end
• To scroll directly to an element

All methods return Bool;

Vertical scroll to # at the top of the page/horizontal scroll to the far left d (scrollable = True). Scroll. ToBeginning () d (scrollable = True). Scroll. Horiz... toBeginning () vertical scroll to the bottom of the page most / # D (scrollable=True).scroll.toend () d(scrollable=True).scroll.horiz.toend ()# Scroll vertically backward to the specified position / Scroll horizontally to the right to the specified position d(scrollable=True).scroll.to(description=" specified position ")d(scrollable=True).scoriz.to (description=" specified position ")# D (scrollable=True).scroll.forward() d(scrollable=True).scroll.forward() ")# scroll until System element appears d(scrollable=True).scroll. To (text="System") Take screenshot of widgetim = d(text="Settings").screenshot()im.save("settings.jpg")Copy the code

5.8 input

5.8.1 Entering user-defined Text

# use ADB broadcast to type D.send_keys ('hello')# Clear the input box d.c. lear_text()Copy the code

5.8.2 Input buttons

The two methods

D. press(' Enter ')# second d.keyevent(' Enter ')Copy the code

Currently, press supports the following buttons

  """        press key via name or key code. Supported key name includes:            home, back, left, right, up, down, center, menu, search, enter,            delete(or del), recent(recent apps), volume_up, volume_down,            volume_mute, camera, power.        """
Copy the code

Keyevent is entered in adb shell input keyevent mode, which supports more diversified keys

More detailed information on key developer.android.com/reference/a…

5.8.3 Switching input methods

# change to UI2 input method, here will hide the original system input method, the default is to use the system input method . The default is Fasled set_fastinput_ime (True) # view the current input method, dc urrent_ime # () return the value (' com. Making. Uiautomator /. FastInputIME ', True)Copy the code

5.8.4 Analog input method function

The functions that can be simulated are Go, Search, Send, Next, done, and Previous.

If the press input key does not work, try this method

# search function d.sin_action ("search")Copy the code

5.9 toast operation

Get_message (timout=5,default='no toast')# Clear the toast cache d.toast.reset()Copy the code

5.10 Monitoring Interface

Use Wather to monitor the interface, which can be used to skip the pop-up box during the test

When Wather is started, a new thread is created to monitor it

You can add more than one Watcher

usage

# Register monitor, when "allow" appears in the interface, Remove ("allow")# remove all monitoring d.watcher.remove()# remove all monitoring d.watcher.remove()# D.watcher.start ()d.watcher.start(2.0) # default monitoring interval 2.0s# force to run all monitoring d.watcher.run()# stop monitoring d.watcher.stop()# stop and remove all monitoring Often used to initialize d.watcher.reset()Copy the code

The watch_context method is now available in version 2.11.0. The watch_context method is much simpler than Watcher. This method is recommended for monitoring and only click() is supported.

WCT = d.watch_context()# monitor allowwct.when ("ALLOW").click()# monitor okwct.when ('OK').click() # enable popover monitor, Wct.wait_stable ()# other implementation code # Stop monitoring wct.stop()Copy the code

5.11 Multi-point sliding

This can be used to achieve pattern unlock

Use the touch kind

Down (x, y)# sleep(x, y)# sleep(x, y)# sleep(x, y)# sleep(x, y) Y)# simulate release touch. Up (x,y)# implement long press, same point press sleep for 5S and then lift d.touch.down(252,1151).sleep(5).up(252,1151)# implement 4 point pattern unlock, D.touch.down (252,1151).move(559,1431).move(804,1674).move(558,1666).up(558,1666)Copy the code

Six, image operation

6.1 screenshot

d.screenshot('test.png')
Copy the code

6.2 Recording a Video

This feeling is a useful feature to record at the beginning of the test case, stop recording at the end, and then if the test fails. Then upload to the test report, perfect restoration of the operation site, the specific principle behind the study.

First, you need to download the dependency, the official recommended using image download:

pip3 install -U "uiautomator2[image]" -i https://pypi.doubanio.com/simple
Copy the code

Perform recording:

# screenRecord ('test.mp4')# screenRecord ('test.mp4')# sleep(10)# screenrecord('test.mp4')Copy the code

6.3 Picture Recognition Click

Download the same set of dependencies as recording video.

This function first manually intercepts the image that needs to be clicked on, and then UI2 matches this image in the interface. Currently, I have tried the accuracy test is not very high, and the error rate is very high, so it is not recommended to use it.

# {'similarity': 0.9314796328544617, 'point': [99, 630]}d.image.match('test.png')Copy the code

7. Application management

7.1 Obtaining APP Information on the Current page

D. app_current()# return () {"package": "com.xueqi.android ", "activity":".common.mainActivity ", "pid": 23007}Copy the code

7.2 Installing Applications

You can download and install the APP from the local path and URL. This method returns no value and raises RuntimeError when the installation fails

Da pp_install # local path installation (' test. The apk) # url da pp_install installation (' http://s.toutiao.com/UsMYE/ ')Copy the code

7.3 Running Applications

By default, running start does not close the application, but keeps the current screen.

To eliminate the previous start state, add the stop=True argument.

Def app_start(self, package_name: STR) def app_start(self, package_name: STR, activity: self, package_name: STR) def app_start(self, package_name: STR, activity: self, package_name: STR) Optional[str]=None, wait: bool = False, stop: bool=False, use_monkey: bool=False): """ Launch application Args: package_name (str): package name activity (str): app activity stop (bool): Stop app before starting the activity. (require activity) use_monkey (bool): use monkey command to start app when activity is not given wait (bool): wait until app started. default False """Copy the code

7.4 Stopping an Application

The difference between stop and clear is that the command used to end an application is different

“Am force-stop” is used.

Clear uses “PM clear”

(' xueqiu.android') d.app_stop("com.xueqiu.android") D.app_clear ('com.xueqiu.android')# End all apps, except excludes D.ap_stop_all (excludes=[' com.xueqio.android '])Copy the code

7.5 Obtaining Application Information

D.app_info ('com.xueqiu. Android ')# {"packageName": "com.xueqiu. Android ", "mainActivity": "Com.xueqiu.android.com mon. Splash. SplashActivity", "label" : "snowball" and "versionName" : "12.6.1", "versionCode" : 257, "size": 72597243}Copy the code

7.6 Obtaining the Application Icon

img = d.app_icon('com.xueqiu.android')img.save('icon.png')
Copy the code

7.7 Waiting for the Application to Start

# wait for the app to become the current app. Return PID. If the startup fails, return 0. Pidd. app_wait('com.xueqiu.android',60,front=True)Copy the code

7.8 Uninstalling an Application

Falsed.app_uninstall('com.xueqiu.android') # excludes= true, does not allow you to install from other poor family background. (Does not allow you to install from other poor family background) # excludes=[],verbose= trueCopy the code

Uninstall the package name list returned by all applications. If the package name list is successfully uninstalled, run the verbose=true command to print the information

uninstalling com.xueqiu.android  OKuninstalling com.android.cts.verifier  FAIL
Copy the code

Or you can modify the source code so that it outputs only successful package names, commented for the added code, and uncommented for the source code

def app_uninstall_all(self, excludes=[], verbose=False): """ Uninstall all apps """ our_apps = ['com.github.uiautomator', 'com.github.uiautomator.test'] output, _ = self.shell(['pm', 'list', 'packages', '-3']) pkgs = re.findall(r'package:([^\s]+)', Output) PKGS = set(PKGS). Difference (our_apps + Excludes) PKGS = list(PKGS) # Add a list of uninstall success #sucess_list = [] for PKg_name in pkgs: if verbose: print("uninstalling", pkg_name, " ", end="", flush=True) ok = self.app_uninstall(pkg_name) if verbose: Print ("OK" if OK else "FAIL") # print("OK" if OK else "FAIL") # sucess_list.append(pkg_name) # return the list of success # return sucess_list return PKGSCopy the code

Eight, other practical methods

8.1 Connecting Devices

# When a PC is connected to only one device, Class Device(_Device, _AppMixIn, _PluginMixIn, _InputMethodMixIn, _DeprecatedMixIn): """ Device object """# for compatible with old codeSession = DeviceCopy the code

Connect () can be connected in the following other ways

# When PC and device are in the same network segment, IP address and port number can be used to connect through WIFI. Connect (" 10.0.1:7912 ") # use default 7912 Portconnect (" http://10.0.0.1 "), connect (" http://10.0.0.1:7912 ") # multiple devices, Connect (" CFF1123EA ") # ADB Device Serial numberCopy the code

8.2 Obtaining Device and Driver Information

8.2.1 Obtaining Driver Information

{"currentPackageName": "com.android.systemui", "displayHeight": 2097, "displayRotation": 0, "displaySizeDpX": 360, "displaySizeDpY": 780, "displayWidth": 1080, "productName": "freedom_turbo_XL", "screenOn": true, "sdkInt": 29, "naturalOrientation": true}Copy the code

8.2.2 Obtaining Device Information

Output all information about the test device, including battery, CPU, memory, etc

D. evice_info# output {"udid": "61C90e6A-BA :1b:ba:46:91:0 E-freedom_Turbo_XL ", "version": "10", "serial": "61c90e6a", "brand": "Schok", "model": "freedom turbo XL", "hwaddr": "ba:1b:ba:46:91:0e", "port": 7912, "sdk": 29, "agentVersion": "0.9.4", "display": {"width": 1080, "height": 2340}, "battery": {"acPowered": false, "usbPowered": true, "wirelessPowered": false, "status": 2, "health": 2, "present": true, "level": 98, "scale": 100, "voltage": 4400, "temperature": 292, "technology": "Li-ion" }, "memory": { "total": 5795832, "around": "6 GB" }, "cpu": { "cores": 8, "hardware": "Qualcomm Technologies, Inc SDM665" }, "arch": "", "owner": null, "presenceChangedAt": "0001-01-01T00:00:00Z", "usingBeganAt": "0001-01-01T00:00:00Z", "product": null, "provider": null}Copy the code

8.2.3 Obtaining the Screen Resolution

# return (width, height) tuple d.window_size()# return (width, height) tuple D.window_size ()# return (width, height) tuple d.window_size()# return (width, height)Copy the code

8.2.4 Obtaining an IP Address

# Return the IP address string, or Noned. Wlan_ip if none is returnedCopy the code

8.3 Driver Global Settings

8.3.1 Setting Using Settings

View the Settings default Settings

D. Settings # output {# delay after click, (0, 3) indicates that the element should wait 0 seconds before clicking, and wait 3S after clicking before executing the subsequent operation 'operation_delay': Swipe/click/press/send_keys/long_click/operation_delay_methods ['click', 'swipe'], # default wait time equivalent to appium implicit wait 'wait_timeout': 20.0, # xpath log 'xpath_DEBUG ': False}Copy the code

To change the default Settings, just modify the Settings dictionary

Settings ['operation_delay'] = (2,4.5)# Settings ['operation_delay_methods'] = {'click','press','send_keys'}# alter default waiting D.sittings ['wait_timeout'] = 10Copy the code

8.3.2 Setting usage or Properties

• HTTP request timeout by default

  Default value: 60s, d.htp_timeout = 60

• When the device is offline, wait for the online duration of the device

  This parameter is valid only when TMQ=true. D. WAIT_FOR_DEVICE_TIMEOUT = 70 can be set using the environment variable WAIT_FOR_DEVICE_TIMEOUT

• Default wait time for element lookup

  If no element is hit, wait 10 days before reporting exception d.implicitly_wait(10.0)

• Example To enable HTTP debugging information, run the following command

  $curl -x POST -d ‘{“jsonrpc”: “2.0”, “id”: “0eed6e063989e5844feba578399e6ff8”, “method”: “deviceInfo”, “params”: {}} ‘http://localhost:51046/jsonrpc/0’ 15:52:04. 816 Response (79 ms) > > > {” jsonrpc “:” 2.0 “, “id” : “0 eed6e063989e5844feba578399e6ff8”, “result” : {” currentPackageName “:” com. Android. Systemui “, “displ ayHeight”:2097,”displayRotation”:0,”displaySizeDpX”:360,”displaySizeDpY”:780,”displayWidth”:1080,”productName”:”freedom_ turbo_XL”,”screenOn”:true,”sdkInt”:29,”naturalOrientation”:true}}<<< END

• dormancy

  Equivalent time. Sleep (10) d.s leep (10)

8.4 bright screen

# creen_on()# creen_off()Copy the code

8.5 Screen Direction

Set_orientation (value) gets the current screen orientationCopy the code

Value A value reference, as long as it is any value in a tuple.

In reality, if you want to achieve this effect, you need to turn the phone 90 degrees counterclockwise (1, "left", "L ", 90),# upside down, (2, "upsidedown", "U ", 180), # right landscape, reverse adjustment to left, screen clockwise 270 degrees (3, "right", "r", 270)Copy the code

8.6 Opening the Notification Bar and Quick Settings

Open notification

d.open_notification()
Copy the code

Turn on quick Settings

d.open_quick_settings()
Copy the code

8.7 File Import or Export

8.7.1 Importing a File

D.ush ("test.txt","/sdcrad/") d.ush ("test.txt","/sdcrad/test.txt")Copy the code

8.7.2 Exporting files

d.pull('/sdcard/test.txt','text.txt')
Copy the code

8.8 Running shell Commands

Execute using shell methods

8.8.1 Running non-blocking Commands

Output returns a whole string. If you need to extract a value, you need to parse and extract the output

Exit_code = d.shell(["ls","-l"],timeout=60)Copy the code

8.8.2 Executing Blocking Commands (Continuously Executing Commands)

# returns a command data flow output requests. The models. The Responseoutput = d.s hell (' logcat, stream = True) try: Iter_lines () : print(line.decode('utf8'))finally: output.close()Copy the code

Source description

def shell(self, cmdargs: Union[str, List[str]], stream=False, timeout=60): "" Run adb shell command with arguments and return its output. Require atx-agent >=0.3.3 Args: cmdargs: str or list, example: "ls -l" or ["ls", "-l"] timeout: seconds of command run, works on when stream is False stream: bool used for long running process. Returns: (output, exit_code) when stream is False requests.Response when stream is True, you have to close it after using Raises: RuntimeError For atx-agent is not support return exit code now. When command got something wrong, exit_code is always 1, otherwise exit_code is always 0 """Copy the code

8.9 Session (now deprecated)

8.10 Stopping THE UI2 Service

Because of atX-Agent, Uiautomator is always guarded and will be restarted if it exits. But Uiautomator is a bully. Once it’s running, accessibility functions on your phone and UIAutomatorViewer on your computer won’t work unless you turn off the framework’s own Uiautomator

Stop using code

d.service("uiautomator").stop()
Copy the code

Manual stop

Directly open the ATX APP (init successfully, it will be installed) and click close UIAutomator

Above, welcome to exchange and discuss together.

Click to collect: Automation + side open + performance + RESUME + interview core tutorial materials