rust-lang / rust-lang/rust

compiletest: `./x test <test-name>.stderr` (or existing non-`.rs` file) should warn or error

Open
#126,601 8 comments 4 reactions 1 assignee View on GitHub

@jieyouxu is already working on this.

Since Sep 7, 2026.

A-compiletest C-bug E-hard T-bootstrap
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

Noticed by @WaffleLapkin, if you write

./x test tests/ui/parser/bad-let-else-statement.stderr

then compiletest will silently pass but run 0 tests, because bad-let-else-statement.stderr is a tracked stderr file associated with the actual .rs ui test but not a test itself. This might be surprising or give a false sense of security to the person who's running the test suite thinking that the test passed, whereas it actually didn't even run.

  • If the path doesn't exist, compiletest will report the non-existent test error.
  • However, if the path does exist, but it's not an actual test (whatever consitutes a "test" file under the relevant test suite), compiletest currently seems to silently ignore and pass.

We should probably warn or error if the path exists (and has an extension) but is not a valid test file.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.