oxidecomputer / oxidecomputer/omicron

sled-agent: transient zone root datasets should be recreated during upgrades

Open
#8,316 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Sled Agent
Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

This is a cousin to #6177, but the problem and fix are both different enough that I think it warrants its own issue. In particular, the "orphaned dataset" work introduced in #8301 does not address transient zone root datasets at all (https://github.com/oxidecomputer/omicron/pull/8301#discussion_r2138624727).

The problem is different for two reasons:

  1. #6177 is a blocker for upgrades because the datasets that we leak share the same name at the ZFS level with new datasets of the same kind on the same zpool, which causes failures when placing new datasets. Transient zone datasets do not have this problem: they include the zone ID in the dataset name at the ZFS level (e.g., oxp_91ea7bb6-2be7-4498-9b0d-a0521509ec00/crypt/zone/oxz_ntp_3ccea933-89f2-4ce5-8367-efb0afeffe97), so a new zone's transient root dataset can still be placed on the same zpool because the zone ID will be different.
  2. These datasets are cleaned up any time sled-agent restarts (in particular, on any sled reboot), because the entire $POOL/crypt/zone/ dataset is destroyed on startup for each pool.

The fix also needs to be different. Durable datasets should only be deleted if their corresponding zone has been expunged. Transient zone roots should be deleted if their corresponding zone has been expunged or if it's being restarted. Failing to do this allows zones to persist data on their root dataset across upgrades, although since the root datasets are not persisted across sled restarts, it's very unlikely any zones are accidentally making use of this.

I think the most straightforward thing to do here is to include deleting the zone's transient root when the zone is shut down, which happens in both the "expunge" and "bounce to upgrade" paths.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in sled-agent's zone shutdown handling and trace both the expunge and bounce-to-upgrade paths. Verify how transient zone root datasets under the zone pool are removed there; done means the transient root is deleted when a zone is expunged or restarted for an upgrade, while durable datasets follow their existing rules.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.