oxidecomputer / oxidecomputer/hubris

Restart Sidecar sequencer task if mainboard controller is reset

Open
#947 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
3.6k
Forks
239
Avg merge
1d 12h
Merged PRs (30d)
23

Description

If some external event or entity resets the mainboard controller FPGA after the sequencer task has started it'll not continue to make forward progress. Consider the following log:

humility: ring buffer drv_sidecar_seq_server::__RINGBUF in sequencer:
 NDX LINE      GEN    COUNT PAYLOAD
   0  442        1        1 FpgaInit
   1  485        1        1 SkipLoadingBitstream
   2  495        1        1 MainboardControllerId(0x1de5bae)
   3  509        1        1 MainboardControllerChecksum(0x6461823f)
   4  541        1        1 MainboardControllerVersion(0x83)
   5  542        1        1 MainboardControllerSha(0x7b53e239)
   6  543        1        1 FpgaInitComplete
   7   25        1        1 LoadingClockConfiguration
   8  564        1        1 ClockConfigurationComplete
   9  568        1        1 FrontIOBoardPresent
  10   58        1        1 FrontIOControllerIdent { fpga_id: 0x0, ident: 0x1deaa55 }
  11   64        1        1 FrontIOControllerChecksum { fpga_id: 0x0, checksum: [ 0x72, 0xa5, 0x6b, 0xec ], expected: [ 0x72, 0xa5, 0x6b, 0xec ] }
  12   78        1        1 SkipLoadingFrontIOControllerBitstream { fpga_id: 0x0 }
  13   58        1        1 FrontIOControllerIdent { fpga_id: 0x1, ident: 0x1deaa55 }
  14   64        1        1 FrontIOControllerChecksum { fpga_id: 0x1, checksum: [ 0x72, 0xa5, 0x6b, 0xec ], expected: [ 0x72, 0xa5, 0x6b, 0xec ] }
  15   78        1        1 SkipLoadingFrontIOControllerBitstream { fpga_id: 0x1 }
  16  581        1        1 FrontIOVsc8562Ready
  17  121        1     5557 TofinoSequencerTick(Disabled, A2, None)
  18  402        1     5579 TofinoSequencerError(FpgaError)

If a TofinoSequencerError is encountered the sequencer should check the device state of the mainboard controller and if it finds the device AwaitingBitstream it should restart itself. The FPGA will have its IOs floating, so nothing on the board works anyway and we may as well restart the task.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the sequencer task's handling of TofinoSequencerError and inspect how it reads the mainboard controller device state. Confirm the behavior using the supplied ring-buffer scenario: when the state is AwaitingBitstream, the sequencer should restart; otherwise it should continue its existing handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
embedded-iot
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.