Refactor system/idempotency tests to use a custom test harness
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7k
- Forks
- 1.1k
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 24
Description
The large amount of system and idempotency tests rustfmt has is great, but each of these test "buckets" run all together as a unit test. As of 3c245a7c3305602ea42b7a3187f5858121c7866b this means we have one unit tests that runs 460 system tests and another that runs 612 idempotency tests. This means there is no convenient way to filter for a single system/idempotency test, making devel of a single feature a bit cumbersome; either
- you have to wait for all system or idempotency tests to run, or
- you run an ad-hoc test on the rustfmt binary, which loses the usefulness of
cargo testfor local devel
My proposal is to refactor the system/idempotency test runner to use a custom test harness (like libtest-mimic). This would allow us to register individual system/idempotency tests and provide compat with all arguments to cargo test, like test filters.
I did this about a month ago for one of my personal projects (https://github.com/yslide/slide/blob/master/slide/src/test/mod.rs), and it's not very difficult. For rustfmt I think it would be as simple as changing the test interface, since the test infra is already set up.
This is low priority.
cc @topecongiro @calebcartwright
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading rustfmt's existing system and idempotency test runner and compare its interface with the libtest-mimic example linked in the issue. Check how the runner is invoked through cargo test and how individual tests are currently grouped. Done means individual system and idempotency tests are registered and cargo test filters and arguments work for them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- testing-qa, tooling
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100