oxidecomputer / oxidecomputer/omicron
Instances shouldn't go into "reincarnation cooldown" if they fail to start through no fault of their own
Open
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
During update testing on dublin, I had an instance enter its reincarnation cooldown (currently 1 hour) through no fault of its own:
- Initially running on sled 17 (had been up for many hours)
- Sled is rebooted for an update; instance is marked as failed when the sled comes back
- Instance is immediately restarted, but placed back on sled 17. It fails due to #9095.
- The instance is now penalized for failing too quickly after start, and must wait for the cooldown to be restarted again.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the instance reincarnation cooldown logic and the failure path described in this issue, then compare the related failure in #9095. Done means failures caused by sled reboot/update handling or that reported fault do not trigger the cooldown, while genuine rapid-start failures still do.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100