google / google/xls

Quickchecks don't execute unless `--compare=jit`

Open
#2,970 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.9k
Forks
283
Avg merge
2d 10h
Merged PRs (30d)
135

Description

**Describe the bug**
Quickchecks don't execute unless `--compare=jit`.

Why would I want to slow down quickcheck execution by running it twice (interpreted and jit), and using the slow interpreter? What if I trust that JIT is correct and I just want to execute my quickcheck 4 billion times, as quickly as possible?

**To Reproduce**

1. Make a file with a quickcheck in it, name it foo.x
2. `dslx_interpreter_main-arm64 --evaluator ir-jit foo.x`, observe: `[ SKIPPING QUICKCHECKS ] (JIT is disabled)`. Also skips quickchecks: `--compare=none --evaluator ir-jit`
4. `dslx_interpreter_main-arm64 --compare=jit foo.x`, observe `RUN QUICKCHECK`
5. Note that if you supply both flags, that's rejected: `--compare flag is only supported on --evaluator=dslx-interpreter`

**Expected behavior**
I would expect that executing quickchecks as quickly as possible (i.e. with evaluator=jit and compare disabled) to be encouraged and accepted. I don't want to compare the interpreter against the JIT, that's just massively slowing things down.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.