oxidecomputer / oxidecomputer/omicron

helios/deploy failed in CI: collections.len() != 1

Open
#10,573 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Test Flake
Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

This test failed on a CI run on https://github.com/oxidecomputer/omicron/pull/10560 (commit 07578fd)

https://github.com/oxidecomputer/omicron/pull/10560/checks?check_run_id=80211588124

i've rerun this, so i'm not sure if the check still has anything useful to offer.

Log showing the specific test failure:

https://buildomat.eng.oxide.computer/wg/0/details/01KTMPA8HJ9Y5XAQ7306QTTZCZ/x4GX4ri3hooCo1wJ54Vy7za25Fw0K5BLFxddqidfDabDXZlK/01KTMPB4PMV7V2KYAEK7PCYZ1Q#S1838

Excerpt from the log showing the failure:

1849  2026-06-09T02:21:18.549Z  ++ pfexec zlogin oxz_switch /opt/oxide/omdb/bin/omdb db zpool list -i
1850  2026-06-09T02:21:18.549Z  ++ wc -l
1851  2026-06-09T02:21:18.580Z  note: database URL not specified.  Will search DNS.
1852  2026-06-09T02:21:18.580Z  note: (override with --db-url or OMDB_DB_URL)
1853  2026-06-09T02:21:18.580Z  note: using DNS from system config (typically /etc/resolv.conf)
1854  2026-06-09T02:21:18.580Z  note: (if this is not right, use --dns-server to specify an alternate DNS server)
1855  2026-06-09T02:21:18.605Z  note: using database URL postgresql://root@[fd00:1122:3344:101::5]:32221,[fd00:1122:3344:101::6]:32221,[fd00:1122:3344:101::7]:32221,[fd00:1122:3344:101::4]:32221,[fd00:1122:3344:101::3]:32221/omicron?sslmode=disable
1856  2026-06-09T02:21:18.630Z  note: database schema version matches expected (264.0.0)
1857  2026-06-09T02:21:21.517Z  Error: Internal Error: failed runtime check: "collections.len() == 1"
1858  2026-06-09T02:21:21.542Z  + ACTUAL_ZPOOL_COUNT='       0'
1859  2026-06-09T02:21:21.542Z  + _exit_trap
1860  2026-06-09T02:21:21.543Z  + local status=1
1861  2026-06-09T02:21:21.543Z  + set +o errexit
1862  2026-06-09T02:21:21.543Z  + [[ x != \x ]]
1863  2026-06-09T02:21:21.543Z  + pfexec svcadm -Z disable -s cron
1864  2026-06-09T02:21:24.330Z  + [[ 1 -eq 0 ]]
1865  2026-06-09T02:21:24.330Z  + set -o xtrace
1866  2026-06-09T02:21:24.330Z  + banner evidence

i've not yet looked at what might have collections.len() == 1'd to have an assertion failure, but i cannot imagine this is related to a new CPU platform...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked Buildomat log and the omdb db zpool list -i command, then trace the source of the collections.len() == 1 runtime check. Determine why the CI deployment reports zero zpool entries and reproduce the failure if possible. Done means the CI test reliably finds the expected single collection or the underlying failure is corrected and covered by an appropriate test.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ci-cd, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.