rust-lang / rust-lang/rust-analyzer

Discover tests using a custom test harness

Open
#21,259 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

C-feature
Dominant language
Rust
Stars
16.9k
Forks
2.2k
Avg merge
1d 12h
Merged PRs (30d)
72

Description

First, thank you for your hard work on rust-analyzer.

We consider migrating from dirtest to datatest-stable. dir-test exports a macro that generates a #[test] function for every file in a given directory. This is great because it plays really nicely with r-a: r-a detects the tests and we can run them by name. However, it has the downside that it requires a build.rs file that invalidates the macro whenever one of our spec files change, which leads to very slow iteration cycles.

That's why we consider migrating to datatest-stable. It uses a custom test harness that mimics cargo test. This is great because it removes the need for the build.rs file and the need to recompile the tests after making changes to the spec files. However, we lose the r-a integration because the tests are now no longer expanded in a macro.

Supporting this use case is tricky because r-a can't make any assumptions about a custom test harness. One possible solution would be to support fetching the known tests using cargo test -- --list (ideally with a custom filter). Obviously, it would fail to automatically detect new tests but that's already the case with our existing dirtest setup where it's necessary to manually clear the macro-cache.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing rust-analyzer's existing test discovery and integration with the dir-test macro. Then investigate how a custom harness exposes tests through cargo test -- --list, including whether a custom filter is possible. Done means custom-harness tests can be discovered and run by name without relying on macro expansion.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
developer-experience, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.