oxidecomputer / oxidecomputer/omicron
[sim] "Not enough datasets" 503 on create disk
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
I don't know when this was introduced bcause I haven't tried to create a disk like this in a while. When I run sim sled agent locally (on Mac — possibly relevant) with `omicron-dev run-things generally work except for creating disks. When I create a disk with the CLI like this:
oxide disk create --project prod-online --json-body /dev/stdin <<EOF
{
"name": "nginx",
"description": "The nginx disk.",
"size": 1073741824,
"disk_source": {
"type": "blank",
"block_size": 512
}
}
EOF
I get a 503 with this internal message in the logs:
saga error at node "datasets_and_regions": Not enough datasets to allocate disks
That error comes from here:
I am not shocked to find there are no regions, zpools, or datasets in the DB:
~/oxide/omicron $ cockroach sql --insecure --url "postgresql://root@127.0.0.1:64043/omicron?sslmode=disable" main
#
# Welcome to the CockroachDB SQL shell.
# All statements must be terminated by a semicolon.
# To exit, type: \q.
#
# Server version: CockroachDB CCL v22.1.9 (x86_64-apple-darwin19, built 2022/10/17 18:39:01, go1.17.11) (same version as client)
# Cluster ID: d494037f-10a5-425a-ae38-bed344cd7148
#
# Enter \? for a brief introduction.
#
root@127.0.0.1:64043/omicron> select * from zpool;
id | time_created | time_modified | time_deleted | rcgen | sled_id | physical_disk_id | total_size
-----+--------------+---------------+--------------+-------+---------+------------------+-------------
(0 rows)
Time: 2ms total (execution 1ms / network 0ms)
root@127.0.0.1:64043/omicron> select * from region;
id | time_created | time_modified | dataset_id | volume_id | block_size | blocks_per_extent | extent_count
-----+--------------+---------------+------------+-----------+------------+-------------------+---------------
(0 rows)
Time: 2ms total (execution 1ms / network 0ms)
root@127.0.0.1:64043/omicron> select * from dataset;
id | time_created | time_modified | time_deleted | rcgen | pool_id | ip | port | kind | size_used
-----+--------------+---------------+--------------+-------+---------+----+------+------+------------
(0 rows)
Time: 4ms total (execution 3ms / network 0ms)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with nexus/db-queries/src/db/queries/region_allocation.rs around the linked allocation logic, then reproduce the issue with the shown oxide disk create command under the sim sled agent and omicron-dev setup. Check the local database state for the missing regions, zpools, and datasets; done means disk creation no longer returns the 503.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100