chore(web): [meta] Test-spec revision for the acceptance tests
- Dominant language
- Pascal
- Stars
- 534
- Forks
- 143
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 113
Description
## Aspect 1 - test suite conversion
The old [acceptance-testing spec for KeymanWeb](https://github.com/keymanapp/keyman/wiki/User-Testing-for-Web) needs conversion to the new [test-spec format](https://github.com/keymanapp/keyman/wiki/User-Testing-Workflows), which is compatible with the user-test bot.
Note: we should _probably_ not test on literally every environment there. A selective cross-section is probably ideal.
My suggestion:
"Standalone mode":
- Windows / Edge (which is Chromium-based)
- macOS / Safari
- Linux / Firefox
- Android 5.0 or 6.0 / default browser
- Android 8.0 or newer / default browser
- iOS / Safari
Note - many of the newer tests may also be good to add to the Android and/or iOS App acceptance test suites, as "embedded mode" tests belong there, rather than here.
## Aspect 2 - extending test coverage
The existing set of tests has fairly limited coverage. All of the OSK work has provided us with a number of examples of additional tests that could be included. That said, do keep in mind that running these tests does take time - try to find a set of tests with both good coverage and good return-on-investment.
Here's a set of notes I drafted with some good reference test suites that may have tests worth including:
#5665 has quite a battery of tests for all forms of the OSK.
#5462 included a number of OSK-styling changes and related tests.
- tests for key-cap scaling
- tests for keyboard docs
- the exact details may be a bit different now, so be sure to run the tests when drafting the acceptance-test spec
- good examples for Android API 21 tests (5.0 / 6.0) vs modern tests (8.0)
Ensure there's a reasonable OSK move + pin test set
- #5409 has a pre-spec version of this.
- Ignore console-command parts, though
- I can't find an actual version of the fully-specced test that uses the current spec format, so my apologies there.
https://github.com/keymanapp/keyman/pull/5387 was the grand gesture-rework PR.
- this has an AWESOMELY detailed analysis of sil_cameroon_qwerty, a keyboard very useful for test-spec writing.
- has fine-tuned detailed spec on nuances of the subkey menu
- some of those details could easily be used for tests that extend our test coverage
Contributor guide
Assessment
This issue has not been assessed yet.