KhronosGroup / KhronosGroup/SPIRV-Tools
[RFC] Add a tool to automatically generate CHECK comments from input SPIR-V
- Dominant language
- C++
- Stars
- 1.4k
- Forks
- 709
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 28
Description
spirv-tools, as well as other open source projects like https://github.com/microsoft/DirectXShaderCompiler, use effcee for unit/regression testing. To create a test out of an expected SPIR-V file, one has to add `; CHECK: ` comments matching the instructions of interest and their dependencies ('inputs'). This is tedious and can be automated, so I'd like to propose a new tool to automatically generate effcee checks.
The LLVM project has a few scripts to generate and update `FileCheck` checks, e.g., https://github.com/llvm/llvm-project/blob/main/llvm/utils/update_test_checks.py. @dneto0 wrote a similar tool for clspv: https://github.com/google/clspv/blob/main/test/spvasm2checks.pl.
To make such a tool usable, we would want to limit the number of `CHECK` lines so that only the instructions of interest and their dependencies are matched (e.g., by following use-def chains). Similarly, the `CHECK`s should not care about the value naming or the exact order of types/constants.
The command line interface could look something like this:
```shell
./generate_checks --match="%foo,%bar,11,:15" [--use-strict-order] [--preserve-control-flow]
```
where `--match` specifies the list of instructions to match (by name, result ID, or line number (in disassembly)), `--use-strict-order` decides whether to use `CHECK` or `CHECK-DAG` for constants and types, and `--preserve-control-flow` decides whether to preserve all control flow constructs.
---
cc: @dneto0, @s-perron, @dnovillo
Contributor guide
Research direction
Start by reading LLVM's update_test_checks.py and clspv's test/spvasm2checks.pl, then review the proposed command-line options and matching rules in this issue. Clarify the intended scope, dependency and ordering behavior, and supported input formats before implementation. Done means the project has an agreed design and a working generator with tests covering the specified behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100