lowRISC / lowRISC/opentitan

[dvsim] Automatically generating reproduction commands in regression logs

Open
#24,314 0 comments 1 reaction 0 assignees View on GitHub
Tool:dvsim Type:Enhancement
Dominant language
SystemVerilog
Stars
3.6k
Forks
1.1k
Avg merge
2d 22h
Merged PRs (30d)
141

Description

### Description

It would be really useful if the failure logs in the regression dashboards (e.g. https://reports.opentitan.org/hw/ip/rv_timer/dv/latest/report.html) could have a button for copying the `dvsim` command that reproduces the failure.

## Example

For example, in that link, the failure message looks like this:

Log message (expand)

![24-08-14_14:37:23](https://github.com/user-attachments/assets/9c4be82a-86b6-4974-810e-893c818a114d)

and there's some useful information about the run at the top:

Run information (expand)

![24-08-14_14:37:03](https://github.com/user-attachments/assets/2eed474a-62b3-4a5b-ad38-9ac113276e0f)

You can put these details (plus the HJSON file path) together to rebuild the reproducing `dvsim` command:

```sh
git checkout 098010d125
./util/dvsim/dvsim.py \
hw/ip/rv_dm/dv/rv_dm_sim_cfg.hjson \
-i rv_timer_stress_all_with_rand_reset \
--build-seed 12185085088694708177096441863424670920996379189869351644310607217057882846251 \
--fixed-seed 2140657422388640192039514280169005108423428466594496183724248505193825521374
```

At lowRISC we triage regressions and often have to create these reproduction commands. It would be really useful if the dashboard could build it automatically for us to copy-paste. This would save a lot of busy-work.

## Extra - access to full logs

As an aside, most of the time we have to reproduce these failures because the logs in the dashboard don't contain enough information. It would be useful to have access to the full logs from `dvsim` generated during the regression runs even if they have to be access controlled. If they already exist, they should be linked to from the dashboard.

Contributor guide

Open the contributing guide

Research direction

Start with util/dvsim/dvsim.py and the example hw/ip/rv_dm/dv/rv_dm_sim_cfg.hjson path to understand the inputs needed for reproduction. Inspect the regression dashboard report linked in the issue and determine where run details and failure logs are assembled. Done means the report exposes a copyable reproducing dvsim command and, where available, links to full logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.