APP Stability Testing

1. Overview

The automated testing framework arkxtest is a key component of the tool suite, supporting JavaScript/TypeScript (JS/TS) unit testing frameworks (JsUnit) and UI testing frameworks (UiTest). JsUnit provides capabilities for executing unit test cases, offers foundational interfaces for test case development, and generates corresponding reports to validate system or application interfaces. UiTest enables users to locate and manipulate UI controls through simple and intuitive APIs, supporting the development of automated test scripts based on interactive UI operations. This guide introduces the framework's core features, implementation principles, environment setup, as well as test script development and execution workflows.

2. Environment Setup

Environment Requirements: Automation script development is primarily based on DevEco Studio; versions 3.0 and later are recommended.

Script execution requires the PC to be connected to the hardware board device.

For DevEco Studio environment setup, please refer to Chapter 3: Application Compilation and Deployment for download instructions and configuration steps.

3. Creating and Writing Test Scripts

To create a new test script, start by creating a new application development project in DevEco Studio. The ohos directory within the project structure serves as the location for test scripts. Navigate to the target module's ETS file under test within the project directory, right-click at any code position, and select Show Context Actions > Create Ohos Test (or use the shortcut Alt+Enter and choose Create Ohos Test) to generate the test class. For detailed instructions, refer to the DevEco Studio documentation.

4. Writing Unit Test Scripts

Note

This chapter primarily focuses on the capabilities supported by the unit testing framework and provides guidance on their utilization.

Within a unit testing framework, test scripts must include the following essential elements:

Dependencies must be imported to utilize the testing interfaces they provide.
Test code development primarily involves implementing testing logic, such as API calls and related operations.
Asserting API calls and setting up checkpoints in test code are critical steps; without these verification points, a test script cannot be considered complete.

The scenario implemented by the following example code involves: launching the test page and verifying whether the device's currently displayed page matches the expected page.

5. Writing UI Test Scripts

This chapter primarily introduces the capabilities supported by the UI testing framework and provides guidance on how to use the corresponding APIs. UI testing builds upon unit testing, with UI test scripts adding calls to the UiTest interface (reference link) on top of unit test scripts to execute specific testing activities. The example code below extends the previous unit test script, implementing a tap operation on the launched application page and subsequently verifying whether the resulting page transition matches the expected outcome.

Add the dependency import: import { Driver, ON } from '@ohos.UiTest';
Writing the index.ets page code

Writing specific test code

Executing the test script Execution in DevEco Studio is triggered by clicking a button, and it currently supports the following execution methods:

Test package-level execution refers to executing all test cases within the test package.
Test suite-level execution refers to executing all test cases defined within the describe method.
Test method-level execution refers to executing the specified it method, i.e., a single test case.

Viewing test results

After test execution, you can directly view the test results in DevEco Studio, as shown in the example figure below:

6. Frequently Asked Questions (FAQ)

Unit Test Cases FAQ

Error 1

The print statements added in the test case are logged after the test results.

Problem Description

The log statements added in the test case did not appear during test execution but were printed only after the test case had completed.

Possible Causes

This situation occurs exclusively in test cases that invoke asynchronous interfaces. By design, all log messages should be printed before the test case concludes execution.

Solution

When multiple asynchronous interfaces are invoked, it is recommended to encapsulate the API calls using a Promise-based approach.

Error 2

Encountered error during test case execution: 'fail to start ability'

Problem Description

During test case execution, the test case failed, and the console returned the error: 'fail to start ability'.

Possible Causes

An issue occurred during the packaging process of the test package, and the test framework dependency files were not included in the test package.

Solution

Verify whether the test package includes the OpenHarmonyTestRunner.abc file. If not found, recompile and repackage the test bundle, then execute the test again.

Error 3

A test case timeout error occurred during execution.

Problem Description

After the test case execution completed, the console displayed an 'execute time XXms' error, indicating that the test case had timed out.

Possible Causes

The test case executed an asynchronous interface, but the done() function was not invoked during execution, causing the test case to hang indefinitely until it timed out.
The function called by the test case took too long to execute, exceeding the configured timeout duration for test case execution.
The test case encountered an assertion failure during the function call, throwing an exception that caused the test execution to hang indefinitely until it timed out.

Solution

Review the test case code logic to ensure that even in assertion failure scenarios, the done() function is executed, thereby guaranteeing the test case completes execution.
You can modify the test case execution timeout configuration parameters in the IDE's Run/Debug Configurations to prevent test case execution timeouts.
Review the test case code logic, validate the assertion results, and ensure all assertions pass.

Common Issues in UI Test Cases

Error 1

The failure logs contain the error messages 'Get windows failed/GetRootByWindow failed'.

Problem Description

The UI test case execution failed, and upon reviewing the HiLog logs, the error messages 'Get windows failed/GetRootByWindow failed' were identified.

Possible Causes

The system's ArkUI switch was not enabled, causing the control tree information for the tested interface to fail to generate.

Solution

Execute the following command, restart the device, and then re-run the test case.

	hdc shell param set persist.ace.testmode.enabled 1
	shell
	hdc shell param set persist.ace.testmode.enabled 1

Error 2

The failure logs contain the error message 'uitest-api does not allow calling concurrently'.

Problem Description

The UI test case execution failed, and upon reviewing the HiLog logs, the error message 'uitest-api does not allow concurrent calls' was identified.

Possible Causes

The UI testing framework's asynchronous interface was not properly called with the 'await' syntax in the test case.
Executing UI test cases in a multi-process manner resulted in multiple UITest processes being launched, which the framework does not support.

Solution

Review the test case implementation and ensure that asynchronous interfaces are properly called using the 'await' syntax.
Avoid executing UI test cases in a multi-process manner.

Error 3

The failure logs contain the error message: 'does not exist on current UI! Check if the UI has changed after you retrieved the widget object'.

Problem Description

The UI test case execution failed, and upon reviewing the HiLog logs, the error message 'does not exist on current UI! Check if the UI has changed after you retrieved the widget object' was identified.

Possible Causes

After the code located the target widget in the test case, the device interface changed, causing the located widget to be lost and preventing subsequent simulated operations.

Solution

Re-run the UI test cases.