Automattic / Automattic/harper
Tests in `harper-core/tests/run_tests.rs` are poor at communicating their expectations
- Dominant language
- Rust
- Stars
- 15.4k
- Forks
- 627
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 102
Description
When they fail, the only thing they communicate is the entire list of lints they got, and that the number of lints doesn't match a magic number they expect.
For most of these tests, this is just an inconvenience. They're short and only expect a few lints, so it's not too difficult to look into them and figure out what they expect. However, there is one test that expects specifically 49 lints. If that fails, well... good luck.
Aside from the difficulty in debugging, this is also a weakness in the tests. Since they're only checking the number of lints, not the type, they can pass even if they get incorrect lints.
Contributor guide
Research direction
Start in harper-core/tests/run_tests.rs and inspect the tests that currently compare lint counts, especially the test expecting 49 lints. Run the existing test suite to observe the failure output, then make the expectations identify the expected lint types rather than only a magic count. Done means failures explain which expectations differ and incorrect lint types cannot pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- developer-experience, testing
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100