clash-lang / clash-lang/clash-compiler
Improve `clash-testsuite` CLI interface
- Dominant language
- Haskell
- Stars
- 1.6k
- Forks
- 169
- Avg merge
- 15h 52m
- Merged PRs (30d)
- 46
Description
It would nice if the CLI for `clash-testsuite` was improved to
* allow easy control over which simulators are used to run the test suite
* provide more effective error reporting when simulators are not in `PATH`
For the first point, I propose adding `--simulator` and `--no-simulator` flags for each simulator we may use in the test suite. These would allow filtering of the tests based on simulator use in a convenient manner. For example
```
$ clash-testsuite --iverilog
```
would only run tests on `verilog`, and only with the `iverilog` simulator.
```
clash-testsuite --installed --no-verilator
```
would run on every simulator on the user's system with the exception of `verilator` (if it is installed). This means where we support multiple simulators for a particular HDL we can limit the jobs to the simulators we care about. The special extra flags `--all` and `--installed` could be used to force all simulators in the test suite / every simulator installed on a user's system.
Regarding reporting: currently when a simulator is not installed and it's tests are executed each test fails with a command not found error in its `stderr`. If we check in advance whether the necessary simulators are installed, we can error early without needing to generate code with `clash`, and provide a single useful error instead of generating HDL then failing for each test, e.g.
```
$ clash-testsuite --iverilog
ERROR: `iverilog` could not be found in PATH.
Please install `iverilog` or choose another simulator.
$ echo $?
127
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the clash-testsuite CLI entry point and trace how simulators are selected, checked, and invoked. Define completion as supporting per-simulator inclusion and exclusion flags plus --all and --installed, with an early, useful PATH error and exit status 127 instead of repeated test failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell
- Domain
- cli, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100