Azure / Azure/azure-sdk-for-rust
Consider supporting cargo-nextest
- Dominant language
- Rust
- Stars
- 884
- Forks
- 365
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 109
Description
As we scale the number of crates in the repo, we'll start to hit some scalability issues. Among them is test run time. Anything that runs a bunch of tests - like `cargo test --workspace --no-fail-fast` - is going to be slow. [cargo-nextest](https://nexte.st) can help, but we'd need to solve a couple problems:
1. It's [configuration](https://nexte.st/docs/configuration/) is centralized. While we can use test patterns to apply defaults for a profile (like a "ci" profile), there's no easy way to build a hierarchy of configuration files like, say, `.gitignore`).
2. This runs more tests in parallel, which we could also accomplish with #4091; however, in local testing I found it not only breaks a bunch of tests, but has some persistent effect:
```bash
cargo test --no-fail-fast # works
cargo nextest run --no-fail-fast # a bunch of tests fail - mostly recorded but, IIRC, not all recorded
cargo test --no-fail-fast # fails, and from a small sample, it's all the same tests as the previous step
```
We also have to make sure `cargo test` keeps working because that's idiomatic and `rustup` nor `cargo` have any keep way to preinstall third-party dependencies like some languages' package managers do (there are some, but they have to be installed first: chicken/egg problem). We could recommend it in `CONTRIBUTING.md`, but then use it in our PRs and CIs with profiles as needed.
Contributor guide
Research direction
Start by reproducing the listed cargo test and cargo nextest commands, including the persistent failures after nextest runs. Read CONTRIBUTING.md and compare the proposed configuration and CI/PR usage with #4091. Done requires a decided approach that preserves cargo test, addresses configuration and test isolation, and specifies any recommendation or CI integration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100