rustdoc doctest UI suite shouldn't need to manually normalize $DIR
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
Summary
I wrote the following UI test:
tests/rustdoc-ui/doctest/force-merge
//@ check-pass
//@ edition: 2018
//@ compile-flags: --test --test-args=--test-threads=1 --merge-doctests=yes -Z unstable-options
//@ normalize-stdout: "tests/rustdoc-ui" -> "$$DIR"
//@ normalize-stdout: "finished in \d+\.\d+s" -> "finished in $$TIME"
//@ normalize-stdout: "ran in \d+\.\d+s" -> "ran in $$TIME"
//@ normalize-stdout: "compilation took \d+\.\d+s" -> "compilation took $$TIME"
//@ normalize-stdout: ".rs:\d+:\d+" -> ".rs:$$LINE:$$COL"
// FIXME: compiletest doesn't support `// RAW` for doctests because the progress messages aren't
// emitted as JSON. Instead the .stderr file tests that this contains a
// "merged compilation took ..." message.
/// ```
/// let x = 12;
/// ```
///
/// These two doctests should be force-merged, even though this uses edition 2018.
///
/// ```
/// fn main() {
/// println!("owo");
/// }
/// ```
pub struct Foo;
In particular, note //@ normalize-stdout: "tests/rustdoc-ui" -> "$$DIR". That was cargo-culted from other tests in rustdoc.
This worked ok locally and generated a .stderr file with
test $DIR/doctest/force-merge.rs - Foo (line 14) ... ok
test $DIR/doctest/force-merge.rs - Foo (line 20) ... ok
It failed in CI because of normalization differences:
- test $DIR/doctest/force-merge.rs - Foo (line 14) ... ok
- test $DIR/doctest/force-merge.rs - Foo (line 20) ... ok
+ test $DIR/force-merge.rs - Foo (line 14) ... ok
+ test $DIR/force-merge.rs - Foo (line 20) ... ok
Note: some mismatched output was normalized before being compared
- test /checkout/tests/rustdoc-ui/doctest/force-merge.rs - Foo (line 14) ... ok
- test /checkout/tests/rustdoc-ui/doctest/force-merge.rs - Foo (line 20) ... ok
+ test $DIR/force-merge.rs - Foo (line 14) ... ok
+ test $DIR/force-merge.rs - Foo (line 20) ... ok
That second note is really weird! Those are absolute paths. I don't see absolute paths when I run rustdoc locally, and I looked at the rustdoc invocation locally and CI and didn't notice any interesting differences. The only thing I can think of is something related to remap-path-prefix, but that job didn't set remap-path-prefix.
I think what happened is that the absolute path got normalized by compiletest, not by the test directive: https://github.com/rust-lang/rust/blob/ab196d7a7de1170ac2b376fb343f633d29d75685/src/tools/compiletest/src/runtest.rs#L2557-L2558
Command used
./x t --stage 1 tests/rustdoc-ui/doctest/force-merge.rs
Expected behaviour
Compiletest normalizes the path to the UI directory automatically.
Actual behaviour
Compiletest normalizes the path in CI, but not locally.
Bootstrap configuration (bootstrap.toml)
profile = "compiler"
change-id = 102579
[rust]
lld = true
deny-warnings = false
download-rustc = true
debug-assertions = false
verbose-tests = true
[build]
description = "\u001b[0;95mlove you love you love you\u001b[0m"
Operating system
macOS Sequoia 15.5 (24F74)
HEAD
ab196d7a7de1170ac2b376fb343f633d29d75685
Additional context
Extracted from https://github.com/rust-lang/rust/pull/149565#issuecomment-3608564732.
@rustbot label T-rustdoc A-testsuite A-doctests
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run ./x t --stage 1 tests/rustdoc-ui/doctest/force-merge.rs and inspect src/tools/compiletest/src/runtest.rs around lines 2557-2558. Compare local and CI normalization of the paths shown in the issue, then verify the test's .stderr output no longer needs the manual tests/rustdoc-ui-to-$DIR directive and is consistent in both environments.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100