google / google/xls

[enhancement] Support multiple test cases for a single test proc

Open
#1,581 0 comments 0 reactions 0 assignees View on GitHub
dslx enhancement testing
Dominant language
C++
Stars
1.9k
Forks
283
Avg merge
2d 10h
Merged PRs (30d)
135

Description

### What's hard to do? (limit 100 words)

When using test procs, it is not possible to run different test cases, that will be eventually listed as separate entries in the report without duplicating test procs. It would be helpful to describe multiple tests cases that can reuse the same `config()` function.

### Current best alternative workaround (limit 100 words)

Currently, the user has to either put all the tests in the single `next()` function, making individual test cases indistinguishable or create multiple test procs, which leads to significantly longer code.

### Your view of the "best case XLS enhancement" (limit 100 words)

It would be great to define many `next` functions, preferably with some additional information that will help identify the test cases in the test summary. Some changes to the procs syntax were proposed in this [comment](https://github.com/google/xls/issues/1284#issuecomment-2232361087), but the example given doesn’t show if specifying multiple test cases for the same design would be possible.

Here are some options for adding multiple test cases to DSLX:

1. Custom attribute for the `next()` function

```rust
#[test_proc]
proc PassthroughTest {
terminator: chan out;
data_s: chan out;
data_r: chan in;

config(terminator: chan out) {
let (data_s, data_r) = chan("data");
spawn Passthrough(data_r, data_s);
(terminator, data_s, data_r)
}

#[test_case(name="First test case")]
next() {
...
send(tok, terminator, true);
}

#[test_case(name="Second test case")]
next(count: u32) {
...
send(tok, terminator, true);
}
}
```

2. Custom attribute with test case name derived from a function name

```rust
#[test_proc]
proc PassthroughTest {
terminator: chan out;
data_s: chan out;
data_r: chan in;

config(terminator: chan out) {
let (data_s, data_r) = chan("data");
spawn Passthrough(data_r, data_s);
(terminator, data_s, data_r)
}

#[test_case]
case_one() {
...
send(tok, terminator, true);
}

#[test_case]
case_two(count: u32) {
...
send(tok, terminator, true);
}
}
```

3. Test cases naming convention taken, for example, from Google Test.

```rust
#[test_proc]
proc PassthroughTest {
terminator: chan out;
data_s: chan out;
data_r: chan in;

config(terminator: chan out) {
let (data_s, data_r) = chan("data");
spawn Passthrough(data_r, data_s);
(terminator, data_s, data_r)
}

test_case_one() {
...
send(tok, terminator, true);
}

test_case_two(count: u32) {
...
send(tok, terminator, true);
}
}
```
One of the challenges for enabling multiple test cases is how to write the `init()` function for each of them. However, I believe that in general `state` for test procs is not required at all, since the `next()` function can be replaced with a for loop.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.