rust-lang / rust-lang/rust

Lint on FileCheck directives with missing colon

Open
#130,981 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-compiletest A-testsuite C-enhancement T-bootstrap
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

In current compiletest -> FileCheck setup (e.g. codegen/assembly tests), if you forgor a colon : in one of the FileCheck directives, it gets silently ignored by both compilest and FileCheck, meaning that you aren't testing anything.

// tests/assembly/selftest.rs

//@ assembly-output: emit-asm

#![crate_type = "lib"]

// CHECK-LABEL: foo:
//-- CHECK-LABEL: bar: # <- make sure this test is actually testing
// CHECK-NOT ret       # <- notice the missing `:`?
#[no_mangle]
pub fn foo(x: u8) {}

This test currently will just silently pass, even though we wanted to check ret doesn't exist (made-up example for illustration purposes). Realized while reviewing https://github.com/rust-lang/rust/pull/128018.

Example missing colon (real example): https://github.com/rust-lang/rust/pull/125626.

We should probably aim to lint on cases like this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the compiletest-to-FileCheck setup and the example in tests/assembly/selftest.rs, then reproduce how a directive without a colon is handled. Trace where FileCheck directives are parsed and determine how the missing-colon case should be reported. Done means the malformed CHECK-NOT example no longer passes silently and a regression test covers the lint.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.