Chapter 6 APP Stability Testing
1. Overview
The automated testing framework arkxtest is a key component of the tool suite, supporting JavaScript/TypeScript (JS/TS) unit testing frameworks (JsUnit) and UI testing frameworks (UiTest). JsUnit provides capabilities for executing unit test cases, offers foundational interfaces for test case development, and generates corresponding reports to validate system or application interfaces. UiTest enables users to locate and manipulate UI controls through simple and intuitive APIs, supporting the development of automated test scripts based on interactive UI operations. This guide introduces the framework's core features, implementation principles, environment setup, as well as test script development and execution workflows.
2. Environment Setup
Environment Requirements: Automation script development is primarily based on DevEco Studio; versions 3.0 and later are recommended.
Script execution requires the PC to be connected to the hardware board device.
For DevEco Studio environment setup, please refer to Chapter 3: Application Compilation and Deployment for download instructions and configuration steps.
3. Creating and Writing Test Scripts
To create a new test script, start by creating a new application development project in DevEco Studio. The ohos directory within the project structure serves as the location for test scripts. Navigate to the target module's ETS file under test within the project directory, right-click at any code position, and select Show Context Actions > Create Ohos Test (or use the shortcut Alt+Enter and choose Create Ohos Test) to generate the test class. For detailed instructions, refer to the DevEco Studio documentation.
4. Writing Unit Test Scripts
Note
This chapter primarily focuses on the capabilities supported by the unit testing framework and provides guidance on their utilization.
Within a unit testing framework, test scripts must include the following essential elements:
Dependencies must be imported to utilize the testing interfaces they provide.
Test code development primarily involves implementing testing logic, such as API calls and related operations.
Asserting API calls and setting up checkpoints in test code are critical steps; without these verification points, a test script cannot be considered complete.
The scenario implemented by the following example code involves: launching the test page and verifying whether the device's currently displayed page matches the expected page.
5. Writing UI Test Scripts
This chapter primarily introduces the capabilities supported by the UI testing framework and provides guidance on how to use the corresponding APIs. UI testing builds upon unit testing, with UI test scripts adding calls to the UiTest interface (reference link) on top of unit test scripts to execute specific testing activities. The example code below extends the previous unit test script, implementing a tap operation on the launched application page and subsequently verifying whether the resulting page transition matches the expected outcome.
1）Add the dependency import: import { Driver, ON } from '@ohos.UiTest';
2）Writing the index.ets page code
3）Writing specific test code
4）Executing the test script Execution in DevEco Studio is triggered by clicking a button, and it currently supports the following execution methods:
Test package-level execution refers to executing all test cases within the test package.
Test suite-level execution refers to executing all test cases defined within the describe method.
Test method-level execution refers to executing the specified it method, i.e., a single test case.
Viewing test results
After test execution, you can directly view the test results in DevEco Studio, as shown in the example figure below:
6. Frequently Asked Questions (FAQ)
Unit Test Cases FAQ
Error 1
The print statements added in the test case are logged after the test results.
Problem Description
The log statements added in the test case did not appear during test execution but were printed only after the test case had completed.
Possible Causes
This situation occurs exclusively in test cases that invoke asynchronous interfaces. By design, all log messages should be printed before the test case concludes execution.
Solution
When multiple asynchronous interfaces are invoked, it is recommended to encapsulate the API calls using a Promise-based approach.
Error 2
Encountered error during test case execution: 'fail to start ability'
Problem Description
During test case execution, the test case failed, and the console returned the error: 'fail to start ability'.
Possible Causes
An issue occurred during the packaging process of the test package, and the test framework dependency files were not included in the test package.
Solution
Verify whether the test package includes the OpenHarmonyTestRunner.abc file. If not found, recompile and repackage the test bundle, then execute the test again.
Error 3
A test case timeout error occurred during execution.
Problem Description
After the test case execution completed, the console displayed an 'execute time XXms' error, indicating that the test case had timed out.
Possible Causes
The test case executed an asynchronous interface, but the done() function was not invoked during execution, causing the test case to hang indefinitely until it timed out.
The function called by the test case took too long to execute, exceeding the configured timeout duration for test case execution.
The test case encountered an assertion failure during the function call, throwing an exception that caused the test execution to hang indefinitely until it timed out.
Solution
Review the test case code logic to ensure that even in assertion failure scenarios, the done() function is executed, thereby guaranteeing the test case completes execution.
You can modify the test case execution timeout configuration parameters in the IDE's Run/Debug Configurations to prevent test case execution timeouts.
Review the test case code logic, validate the assertion results, and ensure all assertions pass.
Common Issues in UI Test Cases
Error 1
The failure logs contain the error messages 'Get windows failed/GetRootByWindow failed'.
Problem Description
The UI test case execution failed, and upon reviewing the HiLog logs, the error messages 'Get windows failed/GetRootByWindow failed' were identified.
Possible Causes
The system's ArkUI switch was not enabled, causing the control tree information for the tested interface to fail to generate.
Solution
Execute the following command, restart the device, and then re-run the test case.
hdc shell param set persist.ace.testmode.enabled 1
shell
hdc shell param set persist.ace.testmode.enabled 1Error 2
The failure logs contain the error message 'uitest-api does not allow calling concurrently'.
Problem Description
The UI test case execution failed, and upon reviewing the HiLog logs, the error message 'uitest-api does not allow concurrent calls' was identified.
Possible Causes
The UI testing framework's asynchronous interface was not properly called with the 'await' syntax in the test case.
Executing UI test cases in a multi-process manner resulted in multiple UITest processes being launched, which the framework does not support.
Solution
Review the test case implementation and ensure that asynchronous interfaces are properly called using the 'await' syntax.
Avoid executing UI test cases in a multi-process manner.
Error 3
The failure logs contain the error message: 'does not exist on current UI! Check if the UI has changed after you retrieved the widget object'.
Problem Description
The UI test case execution failed, and upon reviewing the HiLog logs, the error message 'does not exist on current UI! Check if the UI has changed after you retrieved the widget object' was identified.
Possible Causes
After the code located the target widget in the test case, the device interface changed, causing the located widget to be lost and preventing subsequent simulated operations.
Solution
Re-run the UI test cases.