Negative UI test error annotations
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.2k
- PR merge metrics
- PR metrics pending
Description
I wish there is a negative assertion, saying that here one shall not expect a HELP note.
Originally posted by @dingxiangfei2009 in https://github.com/rust-lang/rust/pull/107251#discussion_r1725678959
Something that might be nice-to-have is negative ui test error annotations, i.e. that we explicitly want to make sure some diagnostic is not emitted (analogous to FileCheck check-NOT annotations).
Obviously, you can rely on checking the blessed stderr, but a negative ui test annotation could help communicate intent in the test itself and is less prone to accidentally letting the blessed stderr though.
Note: we should probably try to make the directive parser more robust and less hacky before attempting to add more syntax.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the UI test directive parser and its existing annotation syntax, as the issue notes that the parser may need to become more robust first. Define how a negative diagnostic assertion should be expressed and validated, then add coverage showing that an unwanted diagnostic is rejected without relying only on blessed stderr.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100