`src/tools/rustdoc-gui-test` uses compiletest directives + `TestProp` handling internals in a questionable way
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
I was wondering why compiletest's Config had a default, it turns out it's used by
to piggy-back off of compiletest's directive parsing and compile/run-flags extraction.
This happens to work because only //@ {compile,run}-flags directives are used, which do not otherwise happen to rely on other parts of the compiletest config or get validated in a way that requires other parts of the compiletest config (for now).
tests/rustdoc-gui/src/extend_css/lib.rs
1://@ compile-flags: --extend-css extra.css
tests/rustdoc-gui/src/theme_css/lib.rs
1://@ compile-flags: --theme custom-theme.css
tests/rustdoc-gui/src/scrape_examples/src/lib.rs
1://@ run-flags:-Zrustdoc-scrape-examples
tests/rustdoc-gui/src/link_to_definition/lib.rs
1://@ compile-flags: -Zunstable-options --generate-link-to-definition
IMO this is very questionable, because this bypasses any validation that compiletest might perform between parsing CLI args passed from bootstrap, and then constructing a validated Config. Furthermore, you'll appear to be able to use //@ only-xxx and other conditional test execution directives, where in reality they do not work at all (or are completely wrong) because the dummy config prevents compiletest conditional test directives from having any info to work with properly.
In other words, compiletest tries to be internally self-consistent / coherent, but it can't enforce that if its internals get exposed and "intercepted".
Possible mitigations / better approaches
- If
//@ {run,compile}-flagsis allrustdoc-gui-testneeds, it might be better to just reimplement a naive per-line//@ {compile,run}-flags. - Or try to merge
rustdoc-gui-testintocompiletest, but it runs tests withnodein quite a different way versus other test suites.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/tools/rustdoc-gui-test/src/main.rs around lines 114-137, then inspect compiletest Config construction and directive parsing. Review the listed rustdoc-gui test files and determine whether the tool should parse only compile/run flags or be integrated with compiletest; done means avoiding an invalid dummy configuration and making unsupported directives unambiguous.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, rust
- Domain
- testing-qa, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100