keymanapp / keymanapp/keyman

chore(web): [meta] Test-spec revision for the acceptance tests

Open
#5,744 0 comments 0 reactions 0 assignees View on GitHub
chore spec web/
Dominant language
Pascal
Stars
534
Forks
143
Avg merge
2d 10h
Merged PRs (30d)
113

Description

## Aspect 1 - test suite conversion

The old [acceptance-testing spec for KeymanWeb](https://github.com/keymanapp/keyman/wiki/User-Testing-for-Web) needs conversion to the new [test-spec format](https://github.com/keymanapp/keyman/wiki/User-Testing-Workflows), which is compatible with the user-test bot.

Note: we should _probably_ not test on literally every environment there. A selective cross-section is probably ideal.

My suggestion:

"Standalone mode":
- Windows / Edge (which is Chromium-based)
- macOS / Safari
- Linux / Firefox
- Android 5.0 or 6.0 / default browser
- Android 8.0 or newer / default browser
- iOS / Safari

Note - many of the newer tests may also be good to add to the Android and/or iOS App acceptance test suites, as "embedded mode" tests belong there, rather than here.

## Aspect 2 - extending test coverage

The existing set of tests has fairly limited coverage. All of the OSK work has provided us with a number of examples of additional tests that could be included. That said, do keep in mind that running these tests does take time - try to find a set of tests with both good coverage and good return-on-investment.

Here's a set of notes I drafted with some good reference test suites that may have tests worth including:

#5665 has quite a battery of tests for all forms of the OSK.

#5462 included a number of OSK-styling changes and related tests.
- tests for key-cap scaling
- tests for keyboard docs
- the exact details may be a bit different now, so be sure to run the tests when drafting the acceptance-test spec
- good examples for Android API 21 tests (5.0 / 6.0) vs modern tests (8.0)

Ensure there's a reasonable OSK move + pin test set
- #5409 has a pre-spec version of this.
- Ignore console-command parts, though
- I can't find an actual version of the fully-specced test that uses the current spec format, so my apologies there.

https://github.com/keymanapp/keyman/pull/5387 was the grand gesture-rework PR.
- this has an AWESOMELY detailed analysis of sil_cameroon_qwerty, a keyboard very useful for test-spec writing.
- has fine-tuned detailed spec on nuances of the subkey menu
- some of those details could easily be used for tests that extend our test coverage

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.