yarn test:unit
Our end-to-end (E2E) testing strategy leverages a combination of technologies to ensure robust test coverage for our mobile applications. We use Wix/Detox for the majority of our automation tests, Appium for specific non-functional testing like app upgrades and launch times, and Bitrise as our CI platform. All tests are written in JavaScript using Jest and Cucumber frameworks.
Detox serves as our primary mobile automation framework, with most of our tests written using it. Learn more about Wix/Detox here.
Supported Platforms: iOS and Android
Test Location: e2e/specs
- Test Wallet: Requires a wallet with access to testnet and mainnet. On Bitrise CI, this wallet is created using a secret recovery phrase from environment variables. For local testing, retrieve the phrase from the
.e2e.env
file. - Environment Variable: Set
IS_TEST='true'
to enable the test environment. Refer to the.e2e.env
file in the mobile 1Password vault for the complete list of environment variables. - Warning Logs: Warning logs may sometimes cause test failures by interfering with automation interactions. To prevent this, disable warning logs during test execution.
- iOS: iPhone 15 Pro
- Android: Pixel 5 API 34
Ensure that these devices are set up. You can change the default devices at any time by updating the device.type
in the Detox config located at e2e/.detoxrc.js
.
-
Start Metro Server: Ensure the Metro server is running before executing tests:
yarn watch:clean
-
Build the Apps for Testing:
-
iOS Debug:
yarn test:e2e:ios:debug:build
-
Android Debug:
yarn test:e2e:android:debug:build
-
-
Run All Tests Locally:
-
iOS Debug:
yarn test:e2e:ios:debug:run
-
Android Debug:
yarn test:e2e:android:debug:run
-
-
Run Specific Tests:
-
iOS:
yarn test:e2e:ios:debug:run e2e/specs/TEST_NAME.spec.js
-
Android:
yarn test:e2e:android:debug:run e2e/specs/TEST_NAME.spec.js
-
-
Run Tests by Tag (e.g., Smoke):
-
iOS:
yarn test:e2e:ios:debug:run --testNamePattern="Smoke"
-
Android:
yarn test:e2e:android:debug:run --testNamePattern="Smoke"
-
Platform: Android
Test Location: wdio
- Default Emulator:
- Name: Android 11 - Pixel 4a API 31
- API Level: 30 (Android 11)
- Configuring Emulator: Update
deviceName
andplatformVersion
inwdio/config/android.config.debug.js
to match your emulator configuration.
-
Create a Test Build:
yarn start:android:qa
-
Run All Tests:
yarn test:wdio:android
-
Run Specific Tests:
yarn test:wdio:android --spec ./wdio/features/Onboarding/CreateNewWallet.feature
Platform: iOS
Test Location: e2e/api-specs/json-rpc-coverage.js
The API Spec tests use the @open-rpc/test-coverage
tool to generate tests from our api-specs OpenRPC Document. These tests are currently executed only on iOS and use the same build as the Detox tests for iOS.
- Test Coverage Tool: The
test-coverage
tool usesRules
andReporters
to generate and report test results. These are passed as parameters in the test coverage tool call located in e2e/api-specs/json-rpc-coverage.js. For more details onRules
andReporters
, refer to the OpenRPC test coverage documentation.
-
Build the App:
yarn test:e2e:ios:debug:build
-
Run API Spec Tests:
yarn test:api-specs
Our CI/CD process is automated through various Bitrise pipelines, each designed to streamline and optimize different aspects of our E2E testing.
- Triggers:
- When "Run Smoke E2E" label is applied to a Pull request: Automatically runs smoke tests.
- Manual Trigger: Select the desired branch in the Bitrise dashboard and choose
pr_smoke_e2e_pipeline
from the pipeline dropdown menu.
- Triggers:
- Nightly: Automatically runs all regression tests against main branch.
- Manual Trigger: Select the main branch (or another branch of choice) in the Bitrise dashboard and choose
pr_regression_e2e_pipeline
from the pipeline dropdown menu.
- Workflows:
- Build: Creates iOS and Android artifacts.
- Test: Executes regression tests across both platforms.
- Manual Trigger: Typically run on release branches but can be manually triggered in the Bitrise dashboard.
- Function: Measures and monitors app launch times on real devices using BrowserStack to ensure consistent performance over time.
- Nightly: Automatically runs on the main branch.
- Manual Trigger: Select the desired branch in the Bitrise dashboard and choose
app_upgrade_pipeline
from the pipeline dropdown menu.
- Function: Automates testing of app upgrades to verify smooth transitions between versions.
- Configuration: Requires the
PRODUCTION_APP_URL
environment variable to be set with the current production build's BrowserStack URL.You would need to search and updatePRODUCTION_APP_URL
in the bitrise.yml with the production browserstack build URL. - Manual Trigger: Select the desired branch in the Bitrise dashboard and choose
app_upgrade_pipeline
from the pipeline dropdown menu.
- Detox Tests: Test reports are displayed directly in the Bitrise UI, offering a visual representation of test results and execution details. Screenshots on test failures are also captured and stored in a zip file. You can download these screenshots from the
Artifacts
tab in Bitrise. - API Spec and Appium Tests: HTML reporters generate and display test results. Access these HTML reports through the Bitrise build artifacts section for detailed analysis.
-
Example:
FAIL e2e/specs/swaps/swap-action-smoke.spec.js (232.814 s) SmokeSwaps Swap from Actions ✓ should Swap .05 'ETH' to 'USDT' (90488 ms) ✕ should Swap 100 'USDT' to 'ETH' (50549 ms) ● SmokeSwaps Swap from Actions › should Swap 100 'USDT' to 'ETH' Test Failed: Timed out while waiting for expectation: TOBEVISIBLE WITH MATCHER(id == “swap-quote-summary”) TIMEOUT(15s) HINT: To print view hierarchy on failed actions/matches, use log-level verbose or higher. 163 | return await waitFor(element(by.id(elementId))) 164 | .toBeVisible() > 165 | .withTimeout(15000); | ^ 166 | } 167 | 168 | static async checkIfNotVisible(elementId) { at Function.withTimeout (e2e/helpers.js:165:8) ...
In this example, the test failed because the
swap-quote-summary
ID was not found. This issue could be due to a changed testID or the swap quotes not being visible. To confirm whether either case is true, we then look at the screenshots on failure.Here we can see that the swaps quotes in fact did not load hence why the tests failed.
- Per Team: Smoke tests are divided by team, allowing targeted verification of core functionalities pertinent to each team's responsibilities.
- Benefits:
- Faster Feedback: Running a subset of tests on PRs provides quicker feedback, ensuring critical functionalities are validated without the overhead of executing all tests.
- Efficient Resource Use: Limits resource consumption and test execution time, optimizing CI/CD pipeline performance.
For more guidelines and best practices, refer to our Best Practices Document.