oxidecomputer / oxidecomputer/omicron
wicket could better handle a trampoline crash during the "Downloading installinator" phase
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
I am not entirely sure how this happened (still investigating), but I appear to have a corrupt trampoline image generated via cargo xtask releng. During the "Downloading installinator, waiting for it to start" phase, this is the output of the serial console on the host:
Oxide Pico Host Boot Loader
Config {
cons: Uart(fedc9000),
loader: 0x7f521000..0x7fff0000
pageroot: P4KA(0x7f52b000),
}
Decompressing cpio archive to 0x77521000..0x7f521000...Done.
jumping to kernel entry at 0xfffffffffbc29da0
Oxide board Gimlet -- GN-B1
Socket 0 SMU Version: 45.93.0
Socket 0 DXIO Version: 45.682
Socket 0 SMU features 0x0690fbfd enabled
cpu0: microcode has been updated from version 0x0 to 0xa0011d3
Oxide Helios Version helios-2.0.22694 64-bit
-----------> Sending IPCC command 0xe, attempt 1/10
Additional data length: 0x3
Received empty frame
Additional data length: 0x1
NOTICE: Starting Oxide boot
NOTICE: Phase 1 wants: a8b525fa731d09310c19bfa5bf68f46d1358e3ea2ff5b0aa454923fb61803a6f
NOTICE: TRYING: boot sp
NOTICE: in image: a8b525fa731d09310c19bfa5bf68f46d1358e3ea2ff5b0aa454923fb61803a6f
NOTICE: received offer from SP --
NOTICE: v2 flags 0x1
NOTICE: data size 0x7873cf2 image size 0x7d000000 target size 0x100000000
NOTICE: dataset rpool/ROOT/ramdisk
NOTICE: image name recovery c2f3515/004716a 2024-05-23 21:07
WARNING: failed ramdisk write at offset 0x5bcdac00%) 163KiB/s
WARNING: failed ramdisk write at offset 0x5bcfac00
-----------> Sending IPCC command 0x6, attempt 1/10
Additional data length: 0x2b
Received empty frame
Additional data length: 0x0
WARNING: Could not find a valid phase2 image on sp
-----------> Sending IPCC command 0x6, attempt 1/10
Additional data length: 0x2b
Received empty frame
Additional data length: 0x0
panic[cpu0]/thread=fffffffffbe9e9a0: Could not find a valid phase2 image on sp
Warning - stack not written to the dump buffer
fffffffffbea4820 boot_image:oxide_boot_locate+18a ()
fffffffffbea4850 genunix:boot_image_locate+a1 ()
fffffffffbea4890 genunix:main+137 ()
fffffffffbea48a0 unix:_locore_start+88 ()
skipping system dump - no dump device configured
rebooting...
Oxide Pico Host Boot Loader
Config {
cons: Uart(fedc9000),
loader: 0x7f521000..0x7fff0000
pageroot: P4KA(0x7f52b000),
}
Decompressing cpio archive to 0x77521000..0x7f521000...Done.
jumping to kernel entry at 0xfffffffffbc29da0
Oxide board Gimlet -- GN-B1
Socket 0 SMU Version: 45.93.0
Socket 0 DXIO Version: 45.682
Socket 0 SMU features 0x0690fbfd enabled
cpu0: microcode has been updated from version 0x0 to 0xa0011d3
Oxide Helios Version helios-2.0.22694 64-bit
-----------> Sending IPCC command 0xe, attempt 1/10
Additional data length: 0x3
Received empty frame
Additional data length: 0x1
NOTICE: Starting Oxide boot
NOTICE: Phase 1 wants: a8b525fa731d09310c19bfa5bf68f46d1358e3ea2ff5b0aa454923fb61803a6f
NOTICE: TRYING: boot sp
NOTICE: in image: a8b525fa731d09310c19bfa5bf68f46d1358e3ea2ff5b0aa454923fb61803a6f
NOTICE: received offer from SP --
NOTICE: v2 flags 0x1
NOTICE: data size 0x7873cf2 image size 0x7d000000 target size 0x100000000
NOTICE: dataset rpool/ROOT/ramdisk
NOTICE: image name recovery c2f3515/004716a 2024-05-23 21:07
received 00000000000f4800 / 0000000007873cf2 ( 0%) 195KiB/s
When the host reboots and starts trying to download the image again, Wicket resets the percentage back to 0% and continues counting up! It would be nice to detect that an error has occurred here (even if only by saying "hey that number is not supposed to go backwards") and inform the user in Wicket, because at minimum any situation where the progress goes backwards is something that needs to be investigated.
[sled 14 00:13:09] Progress 11/17) Downloading installinator, waiting for it to start: 85.70% (108247638/126307570 bytes) after 717.06s
[sled 14 00:13:12] Progress 11/17) Downloading installinator, waiting for it to start: 86.06% (108706392/126307570 bytes) after 720.06s
[ 00:13:12] Status 0/1: 1 running, 0 completed
[sled 14 00:13:15] Progress 11/17) Downloading installinator, waiting for it to start: 86.48% (109233326/126307570 bytes) after 723.06s
[ 00:13:17] Status 0/1: 1 running, 0 completed
[sled 14 00:13:18] Progress 11/17) Downloading installinator, waiting for it to start: 86.90% (109758312/126307570 bytes) after 726.06s
[sled 14 00:13:21] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 729.06s
[ 00:13:22] Status 0/1: 1 running, 0 completed
[sled 14 00:13:24] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 732.06s
[sled 14 00:13:27] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 735.06s
[ 00:13:27] Status 0/1: 1 running, 0 completed
[sled 14 00:13:30] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 738.06s
[ 00:13:32] Status 0/1: 1 running, 0 completed
[sled 14 00:13:33] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 741.06s
[sled 14 00:13:36] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 744.06s
[ 00:13:37] Status 0/1: 1 running, 0 completed
[sled 14 00:13:39] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 747.06s
[sled 14 00:13:42] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 750.06s
[ 00:13:42] Status 0/1: 1 running, 0 completed
[sled 14 00:13:45] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 753.06s
[ 00:13:47] Status 0/1: 1 running, 0 completed
[sled 14 00:13:48] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 756.06s
[sled 14 00:13:51] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 759.06s
[ 00:13:52] Status 0/1: 1 running, 0 completed
[sled 14 00:13:54] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 762.06s
[sled 14 00:13:57] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 765.06s
[ 00:13:57] Status 0/1: 1 running, 0 completed
[sled 14 00:14:00] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 768.06s
[ 00:14:02] Status 0/1: 1 running, 0 completed
[sled 14 00:14:03] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 771.06s
[sled 14 00:14:06] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 774.06s
[ 00:14:07] Status 0/1: 1 running, 0 completed
[sled 14 00:14:09] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 777.06s
[sled 14 00:14:12] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 780.06s
[ 00:14:12] Status 0/1: 1 running, 0 completed
[sled 14 00:14:15] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 783.06s
[ 00:14:17] Status 0/1: 1 running, 0 completed
[sled 14 00:14:18] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 786.06s
[sled 14 00:14:21] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 789.06s
[ 00:14:22] Status 0/1: 1 running, 0 completed
[sled 14 00:14:24] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 792.06s
[ 00:14:27] Status 0/1: 1 running, 0 completed
[sled 14 00:14:27] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 795.06s
[sled 14 00:14:30] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 798.06s
[ 00:14:32] Status 0/1: 1 running, 0 completed
[sled 14 00:14:33] Progress 11/17) Downloading installinator, waiting for it to start: 87.13% (110053434/126307570 bytes) after 801.06s
[sled 14 00:14:36] Progress 11/17) Downloading installinator, waiting for it to start: 0.07% ( 89808/126307570 bytes) after 804.06s
[ 00:14:37] Status 0/1: 1 running, 0 completed
[sled 14 00:14:39] Progress 11/17) Downloading installinator, waiting for it to start: 0.49% ( 615768/126307570 bytes) after 807.06s
[ 00:14:42] Status 0/1: 1 running, 0 completed
[sled 14 00:14:42] Progress 11/17) Downloading installinator, waiting for it to start: 0.85% ( 1072574/126307570 bytes) after 810.06s
[sled 14 00:14:45] Progress 11/17) Downloading installinator, waiting for it to start: 1.27% ( 1598534/126307570 bytes) after 813.06s
[ 00:14:47] Status 0/1: 1 running, 0 completed
[sled 14 00:14:48] Progress 11/17) Downloading installinator, waiting for it to start: 1.68% ( 2123520/126307570 bytes) after 816.06s
[sled 14 00:14:51] Progress 11/17) Downloading installinator, waiting for it to start: 2.10% ( 2649480/126307570 bytes) after 819.06s
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating Wicket's handling of the "Downloading installinator, waiting for it to start" phase and the progress values shown in the report. Reproduce or test the reset from about 87% to 0%, then make the user-facing behavior identify the backwards progress as an error rather than continuing normally.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100