Audit `run-fail` ui tests to see if they need *exact* run stderr/stdout
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
compiletest triage: I'm not convinced switching
error-patternrun-fail ui tests to use only the current form ofcheck-run-resultsis an improvement overerror-patternunconditionally. If anything, it would be a regression to flaky tests in many cases that are sensitive to exact run stdout/stderr (line numbers, backtraces that can depend on platform-specific unwind mechanism, concrete addresses, version numbers, etc.).I would be in favor of auditing some of these run-fail tests and sparingly add
check-run-resultsif the exact run stderr/stdout is critical for the intention of the test, but not in a blanket fashion. As such, I'm going to close this issue in favor of a more concrete audit issue.
Originally posted by @jieyouxu in #65865
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by auditing the Rust compiletest run-fail UI tests that use error-pattern, comparing their intended checks with the current check-run-results behavior. Identify tests where exact run stdout or stderr is essential, especially cases involving line numbers, backtraces, addresses, or version numbers; done means selectively updating only those tests and preserving platform-stable expectations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers, testing-qa
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100