lowRISC / lowRISC/opentitan

[verilator, tests] Failing Verilator test targets

Open
#24,182 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
SystemVerilog
Stars
3.6k
Forks
1.1k
Avg merge
2d 22h
Merged PRs (30d)
141

Description

### Description

All Verilator test targets have been run in public CI in preparation for the addition of a weekly/nightly Verilator job. Of the current 390 targets, 14 tests explicitly failed within their allocated timeout (`--test_timeout=2400,2400,3600,-1`). These failures have also been reproduced when executing the tests locally. Tracking the state of tests that timed out will be covered in a separate issue.

To reproduce any of these failures, after setting up Verilator for Opentitan, run

```sh
./bazelisk.sh test -t- --test_output=streamed //sw/device/tests:
```

Of these 14 tests, the below 8 are failing for identifiable reasons, which will be addressed in a separate PR:

- `pwrmgr_all_reset_reqs_test_sim_verilator`:
- `rv_core_ibex_mem_test_prod_sim_verilator`
- `usbdev_aon_wake_reset_test_sim_verilator`
- `spi_host_winbond_flash_test_sim_verilator`
- `spi_host_irq_test_sim_verilator`
- `spi_host_smoketest_sim_verilator`
- `gpio_smoketest_sim_verilator`
- `example_mem_ujcmd_test_sim_verilator`

The remaining 6 tests are failing for reasons that are not clear to me. We should look into why these tests are failing specifically when run in the Verilator execution environment, and either aim to fix these tests or mark them as broken / not intended to be run in Verilator accordingly.

- [ ] `usbdev_setuprx_test_sim_verilator`
- Error: `usbdev_setuprx_test.c:208] CHECK-STATUS-fail: DeadlineExceeded:["USB",111]`
- [ ] `usbdev_config_host_test_sim_verilator`
- Error: `usbdev_config_host_test.c:98] CHECK-fail: success`
- [ ] `usbdev_logging_test_sim_verilator`
- Error: ```Error: Vchip_sim_tb: ../src/lowrisc_dv_dpi_c_usbdpi_0.1/usbdpi.c:694: void getTestConfig(usbdpi_ctx_t*, uint16_t):
Assertion !"Device is stalled"' failed.
[2024-07-29T23:36:53Z INFO opentitantool] Command result: Broken pipe```
- [ ] `pwrmgr_usb_clk_disabled_when_active_test_sim_verilator`
- Error: `pwrmgr_usb_clk_disabled_when_active_test.c:112] CHECK-fail: Unexpected hung address`
- [ ] `pwrmgr_usbdev_smoketest_sim_verilator`
- Error: ```Error: Vchip_sim_tb: ../src/lowrisc_dv_dpi_c_usbdpi_0.1/usbdpi.c:1208: uint8_t usbdpi_host_to_device(void*, const svBitVecVal*):
Assertion !"USBDPI: no response to Control Transfer" failed.
[2024-07-30T00:41:41Z INFO opentitantool] Command result: Broken pipe```
- [ ] `clkmgr_jitter_frequency_test_sim_verilator`
- Error: `clkmgr_jitter_frequency_test.c:66] CHECK-STATUS-fail: Internal:["CMT",233]`

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the six listed *_sim_verilator targets with the provided bazelisk.sh command and start from the cited C failure locations: usbdev_setuprx_test.c, usbdev_config_host_test.c, usbdev_logging_test.c, pwrmgr_usb_clk_disabled_when_active_test.c, pwrmgr_usbdev_smoketest_test.c, and clkmgr_jitter_frequency_test.c. Compare each failure with its Verilator execution environment and determine whether the test passes after a fix or should be marked unsupported.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
embedded-iot, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.