oxidecomputer / oxidecomputer/hubris

RoT failed to update with UpdateNotStarted

Open
#1,453 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
3.6k
Forks
239
Avg merge
1d 12h
Merged PRs (30d)
23

Description

In updating the rack, we had two sleds (cubby 9 and cubby 26) fail with UpdateNotStarted:

2023-07-05-19-28-12_2188x562

The dump from the SP on sled 26 isn't particularly revealing:

$ humility -d ./hubris.core.sled-26.failed-to-update-rot ringbuf control_plane_agent
humility: attached to dump
humility: ring buffer task_control_plane_agent::__RINGBUF in control_plane_agent:
 NDX LINE      GEN    COUNT PAYLOAD
   4  497    55377       13 Rx(UdpMetadata { addr: Ipv6(Ipv6Address([ 0xfe, 0x80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xaa, 0x40, 0x25, 0xff, 0xfe, 0x5, 0x0, 0x2 ])), port: 0x56ce, size: 0x20, vid: 0x301 })
   5  497    55377        1 Rx(UdpMetadata { addr: Ipv6(Ipv6Address([ 0xfe, 0x80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xaa, 0x40, 0x25, 0xff, 0xfe, 0x5, 0x4, 0x2 ])), port: 0x56ce, size: 0xa, vid: 0x302 })
   6   84    55377        1 MgsMessage(SpState)
   7  497    55377        1 Rx(UdpMetadata { addr: Ipv6(Ipv6Address([ 0xfe, 0x80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xaa, 0x40, 0x25, 0xff, 0xfe, 0x5, 0x4, 0x2 ])), port: 0x56ce, size: 0xe, vid: 0x302 })
   8  836    55377        1 MgsMessage(Inventory)
   9  497    55377        1 Rx(UdpMetadata { addr: Ipv6(Ipv6Address([ 0xfe, 0x80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xaa, 0x40, 0x25, 0xff, 0xfe, 0x5, 0x4, 0x2 ])), port: 0x56ce, size: 0xe, vid: 0x302 })
  10  836    55377        1 MgsMessage(Inventory)
  11  497    55377        1 Rx(UdpMetadata { addr: Ipv6(Ipv6Address([ 0xfe, 0x80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xaa, 0x40, 0x25, 0xff, 0xfe, 0x5, 0x4, 0x2 ])), port: 0x56ce, size: 0xe, vid: 0x302 })
  12  836    55377        1 MgsMessage(Inventory)
  13  497    55377        1 Rx(UdpMetadata { addr: Ipv6(Ipv6Address([ 0xfe, 0x80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xaa, 0x40, 0x25, 0xff, 0xfe, 0x5, 0x4, 0x2 ])), port: 0x56ce, size: 0xe, vid: 0x302 })
  14  836    55377        1 MgsMessage(Inventory)
  15  497    55377        1 Rx(UdpMetadata { addr: Ipv6(Ipv6Address([ 0xfe, 0x80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xaa, 0x40, 0x25, 0xff, 0xfe, 0x5, 0x4, 0x2 ])), port: 0x56ce, size: 0xe, vid: 0x302 })
   0  836    55378        1 MgsMessage(Inventory)
   1  497    55378        1 Rx(UdpMetadata { addr: Ipv6(Ipv6Address([ 0xfe, 0x80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xaa, 0x40, 0x25, 0xff, 0xfe, 0x5, 0x4, 0x2 ])), port: 0x56ce, size: 0xe, vid: 0x302 })
   2  836    55378        1 MgsMessage(Inventory)
   3  497    55378       13 Rx(UdpMetadata { addr: Ipv6(Ipv6Address([ 0xfe, 0x80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xaa, 0x40, 0x25, 0xff, 0xfe, 0x5, 0x4, 0x2 ])), port: 0x56ce, size: 0x20, vid: 0x302 })
humility: ring buffer task_control_plane_agent::update::rot::__RINGBUF in control_plane_agent:
 NDX LINE      GEN    COUNT PAYLOAD
  38  190        8        1 WriteOneBlock(0xec, 0x200, 0x200)
  39  190        8        1 WriteOneBlock(0xed, 0x200, 0x200)
  40  138        8        1 IngestChunkInput { offset: 0x1dd8a, len: 0x3d2 }
  41  161        8        1 IngestChunkState { offset: 0x1dc00, len: 0x18a }
  42  190        8        1 WriteOneBlock(0xee, 0x200, 0x200)
  43  190        8        1 WriteOneBlock(0xef, 0x200, 0x200)
  44  138        8        1 IngestChunkInput { offset: 0x1e15c, len: 0x3d2 }
  45  161        8        1 IngestChunkState { offset: 0x1e000, len: 0x15c }
  46  190        8        1 WriteOneBlock(0xf0, 0x200, 0x200)
  47  190        8        1 WriteOneBlock(0xf1, 0x200, 0x200)
  48  138        8        1 IngestChunkInput { offset: 0x1e52e, len: 0x3d2 }
  49  161        8        1 IngestChunkState { offset: 0x1e400, len: 0x12e }
  50  190        8        1 WriteOneBlock(0xf2, 0x200, 0x200)
  51  190        8        1 WriteOneBlock(0xf3, 0x200, 0x200)
  52  138        8        1 IngestChunkInput { offset: 0x1e900, len: 0x3d2 }
  53  161        8        1 IngestChunkState { offset: 0x1e800, len: 0x100 }
  54  190        8        1 WriteOneBlock(0xf4, 0x200, 0x200)
  55  190        8        1 WriteOneBlock(0xf5, 0x200, 0x200)
  56  138        8        1 IngestChunkInput { offset: 0x1ecd2, len: 0x3d2 }
  57  161        8        1 IngestChunkState { offset: 0x1ec00, len: 0xd2 }
  58  190        8        1 WriteOneBlock(0xf6, 0x200, 0x200)
  59  190        8        1 WriteOneBlock(0xf7, 0x200, 0x200)
  60  138        8        1 IngestChunkInput { offset: 0x1f0a4, len: 0x3d2 }
  61  161        8        1 IngestChunkState { offset: 0x1f000, len: 0xa4 }
  62  190        8        1 WriteOneBlock(0xf8, 0x200, 0x200)
  63  190        8        1 WriteOneBlock(0xf9, 0x200, 0x200)
   0  138        9        1 IngestChunkInput { offset: 0x1f476, len: 0x3d2 }
   1  161        9        1 IngestChunkState { offset: 0x1f400, len: 0x76 }
   2  190        9        1 WriteOneBlock(0xfa, 0x200, 0x200)
   3  190        9        1 WriteOneBlock(0xfb, 0x200, 0x200)
   4  138        9        1 IngestChunkInput { offset: 0x1f848, len: 0x3d2 }
   5  161        9        1 IngestChunkState { offset: 0x1f800, len: 0x48 }
   6  190        9        1 WriteOneBlock(0xfc, 0x200, 0x200)
   7  190        9        1 WriteOneBlock(0xfd, 0x200, 0x200)
   8  138        9        1 IngestChunkInput { offset: 0x1fc1a, len: 0x3d2 }
   9  161        9        1 IngestChunkState { offset: 0x1fc00, len: 0x1a }
  10  190        9        1 WriteOneBlock(0xfe, 0x200, 0x200)
  11  138        9        1 IngestChunkInput { offset: 0x1ffec, len: 0x3d2 }
  12  161        9        1 IngestChunkState { offset: 0x1fe00, len: 0x1ec }
  13  190        9        1 WriteOneBlock(0xff, 0x200, 0x200)
  14  190        9        1 WriteOneBlock(0x100, 0x200, 0x200)
  15  138        9        1 IngestChunkInput { offset: 0x203be, len: 0x3d2 }
  16  161        9        1 IngestChunkState { offset: 0x20200, len: 0x1be }
  17  190        9        1 WriteOneBlock(0x101, 0x200, 0x200)
  18  190        9        1 WriteOneBlock(0x102, 0x200, 0x200)
  19  138        9        1 IngestChunkInput { offset: 0x20790, len: 0x3d2 }
  20  161        9        1 IngestChunkState { offset: 0x20600, len: 0x190 }
  21  190        9        1 WriteOneBlock(0x103, 0x200, 0x200)
  22  190        9        1 WriteOneBlock(0x104, 0x200, 0x200)
  23  138        9        1 IngestChunkInput { offset: 0x20b62, len: 0x3d2 }
  24  161        9        1 IngestChunkState { offset: 0x20a00, len: 0x162 }
  25  190        9        1 WriteOneBlock(0x105, 0x200, 0x200)
  26  190        9        1 WriteOneBlock(0x106, 0x200, 0x200)
  27  138        9        1 IngestChunkInput { offset: 0x20f34, len: 0x3d2 }
  28  161        9        1 IngestChunkState { offset: 0x20e00, len: 0x134 }
  29  190        9        1 WriteOneBlock(0x107, 0x200, 0x200)
  30  190        9        1 WriteOneBlock(0x108, 0x200, 0x200)
  31  138        9        1 IngestChunkInput { offset: 0x21306, len: 0x3d2 }
  32  161        9        1 IngestChunkState { offset: 0x21200, len: 0x106 }
  33  190        9        1 WriteOneBlock(0x109, 0x200, 0x200)
  34  190        9        1 WriteOneBlock(0x10a, 0x200, 0x200)
  35  138        9        1 IngestChunkInput { offset: 0x216d8, len: 0x3d2 }
  36  161        9        1 IngestChunkState { offset: 0x21600, len: 0xd8 }
  37  190        9        1 WriteOneBlock(0x10b, 0x200, 0x200)

Reattempting the upgrade succeeded.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the control_plane_agent::update::rot ring buffer shown in the issue and compare the failed dump with the successful retry. Trace how UpdateNotStarted is reported around the listed write and ingest events. Done would require identifying the cause and a corrective change, but the issue names no source file or test.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
embedded-iot, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.