rust-lang / rust-lang/rust-clippy

Check for possible silently passing invalid tests by making sure some sort of `panic!` exists in tests

Open
#12,484 4 comments 0 reactions 1 assignee View on GitHub

@kayagokalp is already working on this.

Since Oct 21, 2024.

A-lint
Dominant language
Rust
Stars
13.5k
Forks
2.2k
Avg merge
2d 10h
Merged PRs (30d)
32

Description

What it does

Consider the following code piece:

...
#[test]
fn test_with_silent_fail() {
  let result = ...;
  
  if result.is_err() {
     eprintln!("failed to do something");
  }
}

This is recently discovered for a timout use case, the test had fail cases correctly implemented but also had a outer check for timeouts with tokio runtime. If the user is using eprintln! inside a test, I feel like the test should also fail in that block. Otherwise println! should be used instead.

So in the control graph, the block that contains eprintln should contain a panic as well, or eprintln should be a println in my opinion and clippy can enforce this.

Advantage

It helps preventing cases where a test can silently pass, while the intention was to give an error thus fail.

Drawbacks

It might hurt the expressivity a little, as people may want to use eprintln inside a test for other reasons than printing an error.

Example
#[test]
fn test_with_silent_fail() {
  let result = ...;
  
  if result.is_err() {
     eprintln!("failed to do something");
  }
}

Could be written as:

#[test]
fn test_with_silent_fail() {
  let result = ...;
  
  if result.is_err() {
     panic!("failed to do something");
  }
}

or

#[test]
fn test_with_silent_fail() {
  let result = ...;
  
  if result.is_err() {
     println!("failed to do something");
  }
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.