MetaMask / MetaMask/metamask-mobile
[Appium] Investigate iOS deeply nested app structure
- Dominant language
- TypeScript
- Stars
- 3k
- Forks
- 1.7k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 669
Description
### What is this about?
Locating elements in iOS on some of the screens can be a challenge, mostly due to how deeply nested elements become on react native contexts. This is a well known issue on the Appium community with no real solution but for some workarounds. Throughout some investigating, we noted that the most impacted areas are the **main screen** (specially individual tokens) and general **stacked overlays**, elements that are on top of other elements are still rendered in the ViewHierarchy. This affects even more power users as they usually have plenty of tokens (even if not all tokens are displayed at once - we probably do some eager token load to improve UX).
### Different explored approaches
#### 1. Increase the maximum snapshot depth
The first approach is usually to increase the maximum depth Appium should search in via the use of `snapshotMaxDepth` and `customSnapshotTimeout` as XCUITest Capabilities:
```javascript
{
"snapshotMaxDepth": 62,
"customSnapshotTimeout": 50000
}
```
This allows us to see some more elements but still not enough.
#### 2. Modify the app so that containers are not accessible but rather the actual contents of the elements we want to find
As highlighted by Maestro in [Interacting with nested components on iOS](https://docs.maestro.dev/platform-support/react-native#interacting-with-nested-components-on-ios) outer containers should have their accessibility disabled while the inner components remain the same.
As an experiment, [removing accessibility from the Wallet View](https://github.com/MetaMask/metamask-mobile/pull/22736) allowed us to go further and we managed to be able to locate tokens., even though only in accounts with light activity and light token holdings. This makes us believe that there is some work that can be done in order to improve the usability of Appium on our tests.
#### 3. Use Browserstack's grid sampling feature (still being explored by QA)
We can leverage the use of the [Browserstack grid sampling feature](https://www.browserstack.com/docs/app-automate/appium/troubleshooting/deeply-nested-elements-ios?fw-lang=nodejs%2Fwebdriverio) as _"this feature divides the screen into a grid and performs targeted searches at each grid point"_ .
This proves to be a great approach (also allows us to locate tokens on power user like profiles) when running tests on BrowserStack but since Appium doesn't natively support this feature, we're blocked from having this tests running locally in the same exact way.
```javascript
"appium:bstackPageSource": {
"enable": true,
"samplesX": 15,
"samplesY": 15,
"maxDepth": 75
}
```
### Important notes
View Flattening on iOS should improve this via newArchitecture which we're already using for both Android and iOS but there might me some room for improvement there (see references section below, specially - https://github.com/appium/appium/issues/14825#issuecomment-2471051537).
It is important that the setup while investigating this follows a common setup as different parameters can result in unwanted results:
1. Use real devices builds to test (`.ipa`) - `.app` with simulator can produce different results.
2. Use Appium inspector to locate elements
3. Use Browserstack to run the apps - either by using Appium inspector via Desktop app or starting a test run that allows time to locate elements without having that session finished due to a test failure.
4. Use a power user profile (the app generated by the [GH Actions job](https://github.com/MetaMask/metamask-mobile/actions/workflows/build-ios-upload-to-browserstack.yml) is good otherwise the latest uploaded artifact in Browserstack is good enough)
5. Tokens should be locatable
Capabilities used while investigating this issue:
```javascript
{
"appium:app": "bs://", // Your browserstack app url
"appium:platformVersion": "17",
"appium:deviceName": "iPhone 15 Pro Max",
"platformName": "iOS",
"appium:automationName": "XCUITest",
"appium:bundleId": "io.metamask.MetaMask",
"appium:snapshotMaxDepth": 62, // we can keep this value to the maximum allows as the performance drawback is acceptable
"appium:customSnapshotTimeout": 50000,
}
```
### References
- https://github.com/appium/appium/issues/19661
- https://github.com/appium/appium/issues/14825
- https://developer.apple.com/forums/thread/749918
- https://github.com/reactwg/react-native-new-architecture/discussions/110
- https://developer.apple.com/forums/thread/749918
### Next steps
- We should investigate further approach 2 and intentionally shape accessibility in a way that also allows Appium to work at its best.
- Explore a way to make `appium:bstackPageSource` available outside the Browserstack context
### Scenario
_No response_
### Design
_No response_
### Technical Details
_No response_
### Threat Modeling Framework
_No response_
### Acceptance Criteria
_No response_
### Stakeholder review needed before the work gets merged
- [ ] Engineering (needed in most cases)
- [ ] Design
- [ ] Product
- [ ] QA (automation tests are required to pass before merging PRs but not all changes are covered by automation tests - please review if QA is needed beyond automation tests)
- [ ] Security
- [ ] Legal
- [ ] Marketing
- [ ] Management (please specify)
- [ ] Other (please specify)
### References
_No response_
Contributor guide
Research direction
Use a real-device .ipa and the power-user profile from build-ios-upload-to-browserstack.yml; inspect the hierarchy with Appium Inspector and run it on BrowserStack using the listed XCUITest capabilities. Compare the accessibility-shaping approach with making bstackPageSource available outside BrowserStack. Done means tokens and stacked-overlay elements are locatable in the agreed setup, with the chosen approach recorded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ios, react-native, typescript
- Domain
- accessibility, mobile, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100