Transient `monorail` crash during startup

@luqmana noticed a monorail dump sitting in a Sidecar SP.

This appears to be a transient issue, so I'm not too worried, but documenting it here for posterity.

Here's the details:

ID TASK                       GEN PRI STATE
10 monorail                     0   6 FAULT: panicked at 'Could not initialize BSP: PhyCommandError(55247)', task/monorail-server/src/main.rs:88:13 (was: ready)
matt@jeeves ~ () $ h -d /staff/core/rack2/BRM44220012/2023-04-18/hubris.core.0 tasks -sl
humility: attached to dump
system time = 8982
ID TASK                       GEN PRI STATE
10 monorail                     0   6 FAULT: panicked at 'Could not initialize BSP: PhyCommandError(55247)', task/monorail-server/src/main.rs:88:13 (was: ready)
   |
   +--->  0x24030eb0 0x0806f412 userlib::sys_panic_stub
                     @ /hubris/sys/userlib/src/lib.rs:990
          0x24030ee8 0x0806f448 userlib::sys_panic
                     @ /hubris/sys/userlib/src/lib.rs:982
          0x24030ee8 0x0806f44c rust_begin_unwind
                     @ /hubris/sys/userlib/src/lib.rs:1298
          0x24030f08 0x0806cdc2 core::panicking::panic_fmt
                     @ /rustc/95a3a7277b44bbd2dd3485703d9a05f64652b60e/library/core/src/panicking.rs:50
          0x24031000 0x0806a9e0 main
                     @ /hubris/task/monorail-server/src/main.rs:58

This panic happens when we fail to send a command to the PHY using Phy::cmd

Looking at the value that's being sent, it looks like it could only be from here:

const MCB_CFG_BUF_START_ADDR: u16 = 0xd7c7;
...
        // Set the start address
        let addr = MCB_CFG_BUF_START_ADDR + bits.start() / 8;
        self.phy.cmd(addr)?;

This is in tesla.rs, which is a VSC8552 or VSC8504

We have a VSC8504 on the board, so this is likely a transient failure to communicate during startup.

Specifically, the PhyCommandError occurs if we see a value of 0x4000 in the MICRO_CMD PHY register (18G) while waiting for a command. Quoth the datasheet,

Bit 14 = 1 typically indicates an error condition where the squelch patch was not loaded

This failure occurs while loading the patch (write_patch_value), so... bad timing / luck?

The core is available in /staff/core/rack2/BRM44220012/2023-04-18/hubris.core.0.

Start with drv/vsc85xx/src/tesla.rs, especially write_patch_value and the Phy::cmd call near MCB_CFG_BUF_START_ADDR. Inspect the core at /staff/core/rack2/BRM44220012/2023-04-18/hubris.core.0 and the PHY command handling to understand the 0x4000 error during patch loading. The issue does not define a concrete fix or completion criterion.