oxidecomputer / oxidecomputer/omicron

Can attach 12 disks to instance (11 local, 1 distributed) but VM won't start

Open
#10,456 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

local storage
Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

Theoretically, we support attaching 12 disks to an instance.
However, in the latest run on Madrid (where there is plenty of space to allocate local disks), I saw that an instance with 12 disks (11 of those being local) won't boot up.
Checking the error message, we get:

"Insufficient capacity: Failed to place instance: Could not find any valid sled on which this instance can be placed", 

Switched to 1 GiB disks instead (surely, 1 GiB local disks should be doable), but I still got the same message.
@leftwo mentioned there's a condition for local disks to be placed in different physical disks, so even if there is enough storage (as a whole), if the number of physical disks is less than that then the request fails.

Maybe if we are already aware of this constraint, the error message could make a reference to that, since this only gets checked after we attempt booting the VM.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing instance placement for a request with 11 local disks and one distributed disk, then inspect the capacity error path shown in the issue. Confirm how the physical-disk constraint is checked after boot placement and make the failure explain that constraint; reproduce the 12-disk case to verify the resulting message.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.