google / google/xls

Unify IR-based tests

Open
#216 0 comments 0 reactions 0 assignees View on GitHub
cleanup ir testing
Dominant language
C++
Stars
1.9k
Forks
283
Avg merge
2d 10h
Merged PRs (30d)
135

Description

Right now we have a bunch of IR tests which are scattered about a few places and often duplicated. Examples:

https://github.com/google/xls/blob/main/xls/codegen/combinational_generator.cc
https://github.com/google/xls/blob/main/xls/codegen/pipeline_generator_test.cc
https://github.com/google/xls/blob/main/xls/interpreter/ir_evaluator_test.cc
https://github.com/google/xls/blob/main/xls/interpreter/proc_interpreter_test.cc

In each of the above files, each unit tests builds up some IR and then runs some tests against it. These actions include:
* Build (System)Verilog using some generator (combinational/pipeline) and compare against a golden reference file.
* Run using the interpreter/JIT.
* Simulate the generated Verilog.

Each of the above test files only performs some subset of the actions above. The codegen tests generally don't run the IR interpreter or JIT. The interpreter/JIT tests don't run codegen. The result is that we duplicate testing the same IR in some places, and then also don't test a piece of IR as extensively as we could in others. It'd be better to have these tests in one place and then run everything against them. These actions would include:

* Run interpreter/JIT on IR (before optimization pipeline).
* Codegen multiple ways (combinational, pipeline) and compare against golden reference (before optimization pipeline).
* Run optimization pipeline.
* Run interpreter/JIT on IR after optimization.
* Codegen multiple ways (combinational, pipeline) after optimization and compare against golden reference.

As an additional bonus we could do quickcheck style testing in each of the tests.

Each of the test actions above could be optionally disabled if necessary.

We have something like this style of test already under the following directory: https://github.com/google/xls/tree/main/xls/tests . Here they do codegen and JIT/interpreter testing. However this is only a subset of the above actions. So a TODO for this bug would be:

1. Implement the above actions in some method of IrTestBase.
2. Migrate tests from the above files into some place under xls/tests and have them use the method from 1.

Might want to keep some of the above tests files around (e..g, combinational_generator_test) to hold some tests which test specific corner cases for that library, but generally the bulk of the testing would be in a shared unified place.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.