[ci,fpga] Identify and fix the cause of flaky FPGA manuf tests
@nsw-lowrisc is already working on this.
Since Aug 6, 2026.
- Dominant language
- SystemVerilog
- Stars
- 3.6k
- Forks
- 1.1k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 141
Description
PR https://github.com/lowRISC/opentitan/pull/30912 disables a set of unstable and flaky manufacturing tests entirely to increase the stability of the CI. Currently the tests are completely disabled.
While disabling the tests in CI brings short-term relieve, the underlying issue has to be identified and fixed.
Various smaller efforts have been made towards the identification or the solution of this problem; #29555 captures one of them. So far we see that the flakiness is directly tied to the use and mode switch of the JTAG port. We could further identify that if such an issue arises, a simple reset (using the FPGA's button reset) is not enough to recover functionality leading to the possibility of some non-resettable state that preserves the issue between runs. A bitstream reload (and the following global reset) clears the issue.
We started (automated) systematic data collection of the CI runs. These efforts are collected in this sheet.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.