Tracking bug for testing improvements
- Dominant language
- Dart
- Stars
- 224
- Forks
- 94
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 2
Description
### CI improvements (reduce time it takes for Github actions to run):
- [x] DWDS tests should be sharded into groups (currently they take much longer than the other CI actions)
- Completed in https://github.com/dart-lang/webdev/pull/1851
- [x] Move tests with a large timeout limit (eg, the [`devtools _test`](https://github.com/dart-lang/webdev/blob/80f8671d5db1d4ee9577f8f407aa64f5697b1dea/dwds/test/devtools_test.dart#L5) at 5 minutes) to a cron job (tracked in https://github.com/dart-lang/webdev/issues/1846)
- [x] All tests should have a timeout set
- Completed in https://github.com/dart-lang/webdev/pull/1854
### Refactor `TestContext` setup:
- [ ] Make `TestServer` more re-usable (described in https://github.com/dart-lang/webdev/issues/1779)
- [x] Make it clearer what parameters should be passed to `TestContext` (eg, it is unclear the difference between `path` vs `pathToServe`). Also the default Dart entry file is in `append_body`, but the default HTML entry file is in `hello_world`.
- Completed in https://github.com/dart-lang/webdev/pull/1844 and https://github.com/dart-lang/webdev/pull/1856
- [ ] Currently a lot of the fields in `TestContext` are `late`, because they are only initialized after `TestContext.setUp()` is called. I think it would make more sense to have two separate classes for test setup: `TestPackage` which is equivalent to `TestContext` , and `TestContext` which is equivalent to `TestContext.setUp()`.
### Other improvements:
- [x] Callstack tests break if test `fixtures` are formatted (described in https://github.com/dart-lang/webdev/issues/1831)
- Fixed in https://github.com/dart-lang/webdev/pull/2124
- [ ] All packages should have a `mono_pkg.yml` file, so that the `analyze` and `format` steps can be run against them during CI
- [x] Support Dart 3.0 breaking changes in tests (tracked in https://github.com/dart-lang/webdev/issues/1818)
- [ ] Switch to using `puppeteer` for all integration tests (tracked in https://github.com/dart-lang/webdev/issues/1847)
- [ ] `dwds/test/fixtures` directory contains dill files (used for testing) along with `utilities.dart` (test utilities) and `context.dart` (test setup). This should be reorganized.
Contributor guide
Research direction
Review the remaining unchecked items, especially the TestServer and TestContext refactor described in issue 1779, the missing mono_pkg.yml files, and the reorganization of dwds/test/fixtures. Start with the linked issues and the existing test setup files; done means each remaining checklist item is completed or has a clear tracking issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- dart, github-actions
- Domain
- ci-cd, testing, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100