Audit {`check-run-results`, `regex-error-pattern`, `error-pattern`, `check-stdout`, `normalize-*`, `dont-check-compiler-*`, `run-rustfix`, `rustfix-only-machine-applicable`, `forbid-output`} family of directives
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
I don't know about other contributors, but I have to look at the source implementation (or trial-and-error) every time I see or try to use one of these compiletest directives. IMO, we should audit the design of these directives, and possibly revamp them entirely.
Concrete confusions:
check-run-resultscheck both run stderr and stdout and puts them into snapshot files (on bless), then compares the subsequent run stderr and stdout against the snapshot.error-patterndoesn't only check stderr (whose stderr? compiler? run?), it can also check stdout (or both??) depending oncheck-stdout,dont-check-compiler-*, and also it can check also compiler stderr or stdout I think??normalize-*(that is notnormalize-stdoutornormalize-stderr) I believe can simultaneously apply to {compiler,run} {stderr,stdout}.run-rustfixwill run rustfix and try to apply all non-placeholder suggestions, including non-machine-applicable ones likeMaybeIncorrectones.rustfix-only-machine-applicableis likerun-rustfixbut only tries to applyMachineApplicablesuggestions.forbid-outputis likeerror-patternbut named completely differently. I don't remember which output pattern of {compiler,run}x{stderr,stdout} it is forbidding.
EDIT:
regex-error-patternis likeerror-patternbut accepts a regex...
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the compiletest directive implementations for check-run-results, error-pattern, regex-error-pattern, check-stdout, normalize-, dont-check-compiler-, run-rustfix, rustfix-only-machine-applicable, and forbid-output. Trace how each directive selects compiler or run output and how snapshots or rustfix are handled. Done means the directive behavior and naming are audited and the issue's listed ambiguities have a decided, documented resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers, testing-qa
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100