oxidecomputer / oxidecomputer/omicron

want way to read data from a Disk through the API

Open
#5,824 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement storage
Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

I believe that right now, if I wanted to download a Disk, I would need to launch a VM with the Disk attached, and a guest image that will let me read the Disk data. That'll function, but it'd be nice to be able to download bytes from the Disk directly.

Crucible Pantry has the /crucible/pantry/0/volume/{id}/bulk-read endpoint, but it's not wired up to anything in the API right now.

I believe that we would need

  • a way to put the Disk into an "export" state, a state I just invented, to tell pantry to attach to the Disk in read-mode
  • an endpoint that maps to pantry's bulk-read to read a range of blocks from the Disk
  • a way to take the Disk out of "export" state, to tell pantry to detach

It may also be possible to implementing reading data from an Image, but this would need some extra logic to decide when pantry attaches/detaches from the Image. Though without a convenience in the API, you could get the job done by creating a Disk from an Image, exporting the Disk, and then deleting it afterwards.

It would be pretty convenient if this exposed the Disk as an endpoint you can GET to download the whole file, while also responding to standard HTTP range-requests. That, instead of having to make multiple requests with a bespoke interface and put it back together client side (which would be similar to what uploading does). But I don't yet know enough about the API workings to know if that's feasible; I suppose if there's limits on API response sizes then it maybe isn't.

I'm intentionally vague saying "a way" for the disk states. I've noticed we already have 3 endpoints that all serve the role of transitioning a disk between pantry-related states

  • /v1/disks/{disk}/bulk-write-start
  • /v1/disks/{disk}/bulk-write-stop
  • /v1/disks/{disk}/finalize

I'm not sure if we'd want to keep tacking on extra state-transition endpoints (bulk-read-start, bulk-read-stop) or do something else

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing /v1/disks/{disk}/bulk-write-start, bulk-write-stop, and finalize endpoints, then compare their state transitions with Crucible Pantry's /crucible/pantry/0/volume/{id}/bulk-read endpoint. Define the Disk export lifecycle and whether range reads or standard HTTP range requests are supported. Done means the API shape, state transitions, and read behavior are agreed and covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.