rust-lang / rust-lang/rustc-dev-guide

Guide answers "what" but not "why" and "how" for codegen tests

Open
#1,965 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-codegen A-FileCheck A-test-suite C-enhancement E-hard T-compiler
Dominant language
HTML
Stars
1.9k
Forks
613
Avg merge
17h 24m
Merged PRs (30d)
39

Description

The section on codegen tests is bereft of an explanation of how to use FileCheck, and I do not think the documentation for FileCheck is the best example for it. The documentation for FileCheck focuses on its usage as a CLI tool first, but we are not passing random args to FileCheck!

In fact, it's probably exactly the reverse order for what we want. What is probably most important is how and when to use the regex capturing, introduced at the end, since most tests shouldn't break if a variable changes its name but uses a few different ones: we don't want to focus on LLVM's exact behavior like that. Then we usually want people to use CHECK-LABELs, again, introduced just before regex usage. Then we want to discuss when to use CHECK-NEXT, CHECK-NOT, and CHECK-DAG, and when not to use them.

In our actual usage, CHECK, the directive they introduce first and focus on using, is mostly a directive for corraling CHECK-NEXT and CHECK-DAGs when CHECK-LABEL does not suffice for that, as CHECK is itself the hardest-to-contain directive.

A step-by-step explanation for

  • how to write an okay FileCheck test
  • how to write a really good one

would be better.

And neither of these answer why we use them, which is particular to the Rust codebase. We can't simply upstream all our tests unless LLVM adds rustc to the test suite, as we want e.g. our MIR opts to be taken into account when running our codegen tests.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked codegen tests section and review its current FileCheck guidance alongside the FileCheck usage it describes. Update the guide with step-by-step examples, guidance on directives and regex captures, and the Rust-specific reasons for these tests; done means the section explains both how to write good tests and why rustc uses them.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers, documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.