[verilator, tests] Failing Verilator test targets
Nobody has claimed this yet.
- Dominant language
- SystemVerilog
- Stars
- 3.6k
- Forks
- 1.1k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 141
Description
### Description
All Verilator test targets have been run in public CI in preparation for the addition of a weekly/nightly Verilator job. Of the current 390 targets, 14 tests explicitly failed within their allocated timeout (`--test_timeout=2400,2400,3600,-1`). These failures have also been reproduced when executing the tests locally. Tracking the state of tests that timed out will be covered in a separate issue.
To reproduce any of these failures, after setting up Verilator for Opentitan, run
```sh
./bazelisk.sh test -t- --test_output=streamed //sw/device/tests:
```
Of these 14 tests, the below 8 are failing for identifiable reasons, which will be addressed in a separate PR:
- `pwrmgr_all_reset_reqs_test_sim_verilator`:
- `rv_core_ibex_mem_test_prod_sim_verilator`
- `usbdev_aon_wake_reset_test_sim_verilator`
- `spi_host_winbond_flash_test_sim_verilator`
- `spi_host_irq_test_sim_verilator`
- `spi_host_smoketest_sim_verilator`
- `gpio_smoketest_sim_verilator`
- `example_mem_ujcmd_test_sim_verilator`
The remaining 6 tests are failing for reasons that are not clear to me. We should look into why these tests are failing specifically when run in the Verilator execution environment, and either aim to fix these tests or mark them as broken / not intended to be run in Verilator accordingly.
- [ ] `usbdev_setuprx_test_sim_verilator`
- Error: `usbdev_setuprx_test.c:208] CHECK-STATUS-fail: DeadlineExceeded:["USB",111]`
- [ ] `usbdev_config_host_test_sim_verilator`
- Error: `usbdev_config_host_test.c:98] CHECK-fail: success`
- [ ] `usbdev_logging_test_sim_verilator`
- Error: ```Error: Vchip_sim_tb: ../src/lowrisc_dv_dpi_c_usbdpi_0.1/usbdpi.c:694: void getTestConfig(usbdpi_ctx_t*, uint16_t):
Assertion !"Device is stalled"' failed.
[2024-07-29T23:36:53Z INFO opentitantool] Command result: Broken pipe```
- [ ] `pwrmgr_usb_clk_disabled_when_active_test_sim_verilator`
- Error: `pwrmgr_usb_clk_disabled_when_active_test.c:112] CHECK-fail: Unexpected hung address`
- [ ] `pwrmgr_usbdev_smoketest_sim_verilator`
- Error: ```Error: Vchip_sim_tb: ../src/lowrisc_dv_dpi_c_usbdpi_0.1/usbdpi.c:1208: uint8_t usbdpi_host_to_device(void*, const svBitVecVal*):
Assertion !"USBDPI: no response to Control Transfer" failed.
[2024-07-30T00:41:41Z INFO opentitantool] Command result: Broken pipe```
- [ ] `clkmgr_jitter_frequency_test_sim_verilator`
- Error: `clkmgr_jitter_frequency_test.c:66] CHECK-STATUS-fail: Internal:["CMT",233]`
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run the six listed *_sim_verilator targets with the provided bazelisk.sh command and start from the cited C failure locations: usbdev_setuprx_test.c, usbdev_config_host_test.c, usbdev_logging_test.c, pwrmgr_usb_clk_disabled_when_active_test.c, pwrmgr_usbdev_smoketest_test.c, and clkmgr_jitter_frequency_test.c. Compare each failure with its Verilator execution environment and determine whether the test passes after a fix or should be marked unsupported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- embedded-iot, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100