rust-lang / rust-lang/rust

`cargo test --doc -- --ignored` appears to build and run examples anotated with ```ignore but doesn't, or at least it does not report any failures.

Open
#149,211 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-doctests C-bug T-rustdoc
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

Problem

Given a documentation example that does not build (or builds but fails execution), annotated with ```ignore, I expect:

  1. Running cargo test --doc reports the example as ignored. No failures are reported. => Correct.
  2. Running cargo test --doc -- --ignored builds (and runs) the sample and reports the build (or execution) errors => Incorrect.
    a. The example is reported as built and run.
    b. The example is reported as succesful.
  3. (Surprisingly?), running cargo test --doc -- --ignored --show-output has the expected behaviour. => Correct
    a . The example is reported as built (or built and run).
    b. Build (or execution) failures are reported.
Steps

Given the following src\lib.rs

/// # Example
/// ```ignore
/// let x = UnknownType::new();
/// println!("Doesn't build");
/// ```
pub fn no_build(left: u64, right: u64) -> u64 {
    left + right
}

/// # Example
/// ```ignore
/// assert!(false);
/// println!("Builds but fails");
/// ```
pub fn build_no_run(left: u64, right: u64) -> u64 {
    left * right
}

/// # Example
/// ```ignore
/// println!("Builds and succeeds");
/// ```
pub fn succeeds(left: u64, right: u64) -> u64 {
    left - right
}

and Cargo.toml

[package]
name = "bar"
version = "0.1.0"
edition = "2024"

[dependencies]

  1. Run cargo test --doc -- --ignored
  2. See (unexpected) output
         Doc-tests bar
    
    running 3 tests
    test src\lib.rs - build_no_run (line 11) ... ok
    test src\lib.rs - no_build (line 2) ... ok
    test src\lib.rs - succeeds (line 20) ... ok
    
    test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
    
  3. Run cargo test --doc -- --ignored --show-output
  4. See (expected) output
        Doc-tests bar
    
    running 3 tests
    test src\lib.rs - no_build (line 2) ... FAILED
    test src\lib.rs - succeeds (line 20) ... ok
    test src\lib.rs - build_no_run (line 11) ... FAILED
    
    successes:
    
    successes:
        src\lib.rs - succeeds (line 20)
    
    failures:
    
    ---- src\lib.rs - no_build (line 2) stdout ----
    error[E0433]: failed to resolve: use of undeclared type `UnknownType`
     --> src\lib.rs:3:9
      |
    3 | let x = UnknownType::new();
      |         ^^^^^^^^^^^ use of undeclared type `UnknownType`
    
    error: aborting due to 1 previous error
    
    For more information about this error, try `rustc --explain E0433`.
    Couldn't compile the test.
    ---- src\lib.rs - build_no_run (line 11) stdout ----
    Test executable failed (exit code: 101).
    
    stderr:
    
    thread 'main' panicked at src\lib.rs:3:1:
    assertion failed: false
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    
    failures:
        src\lib.rs - build_no_run (line 11)
        src\lib.rs - no_build (line 2)
    
    test result: FAILED. 1 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.12s
    
    error: doctest failed, to rerun pass `--doc`
    
    
Possible Solution(s)

No response

Notes

No response

Version
cargo 1.88.0 (873a06493 2025-05-10)
release: 1.88.0
commit-hash: 873a0649350c486caf67be772828a4f36bb4734c
commit-date: 2025-05-10
host: x86_64-pc-windows-msvc
libgit2: 1.9.0 (sys:0.20.0 vendored)
libcurl: 8.12.1-DEV (sys:0.4.80+curl-8.12.1 vendored ssl:Schannel)
os: Windows 10.0.22631 (Windows 11 Enterprise) [64-bit]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the documented src/lib.rs and Cargo.toml using cargo test --doc -- --ignored and compare it with --show-output. Inspect the doctest handling behind these command entry points; done means ignored examples are built and executed when requested, with compilation or execution failures reported consistently without requiring --show-output.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.