rust-lang / rust-lang/rust

libtest filters should include the test crate's name in the string matched against the filter

Open
#141,862 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-libtest C-discussion T-libs T-testing-devex
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

Let's say I have a whole bunch of .rs files in my tests folder, like:

  • floats/lib.rs
  • time.rs
  • thread.rs
  • ...

Now I want to run just the float tests. Naturally I'll do something like cargo test -- float. Sadly that does not work since this runs the tests that have float as a substring in the name of the test (mod1::mod2::test_fn), but the name of the crate is not included.

This is particularly confusing in the rustc repo itself, where ./x test library/{core,std} -- float will run the float tests in libcore but not the one in libstd: libcore has a single large test crate with a module called floats, while std has a bunch of separate test crates, one of which is called floats.

I think libtest should be changed to put the crate name in the string used for filtering, thus avoiding this confusing and surprising behavior.

@rust-lang/libs-api I've been told the libtest CLI API is in your purview. Is there any chance the behavior of filters could be adjusted? This would lead to strictly more tests being included in any given filter, which at least won't silently reduce test coverage (except when combined with --skip) -- but it could theoretically break a setup where that then includes a test that does not actually work.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the libtest filtering implementation and reproduce the issue using separate integration-test crates such as floats/lib.rs and time.rs, running cargo test -- float. Trace how test names are assembled for filtering; done means the test crate name participates in matching without silently excluding existing matches, including the documented --skip caveat.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.