No disk-preserving recovery for a storage-wedged sprite: restore unreachable, no stop/restart API, documented DELETE exec returns 404

Open
#231 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Domain
api, cli, documentation

Research direction

Start by comparing the documented DELETE exec endpoint and available CLI/API recovery operations with the reported rc47/rc48 behavior. Reproduce the checkpoint, exec-delete, idle-suspend, and container-stop cases; done means a disk-preserving recovery path exists, wedged requests can time out without blocking suspension, and the endpoint documentation matches reality.

Written by the indexing model from the issue text.

Description

Companion to the disk-I/O-wedge report in #230. When a sprite's storage wedges while the VM stays running, every disk-preserving remedy turns out to be unreachable, and the owner's only exit is sprite destroy — losing checkpoints, credentials, caches, everything. Verified on supervisor 0.0.1-rc47 (2026-08-24) and again on rc48 (2026-09-01), org junchao-yan:

  1. Checkpoint restore routes through the wedged overlay manager. GET /v1/sprites/{name}/checkpoints hangs forever, and restore needs the same manager — so the one mechanism designed to recover a broken overlay is unavailable precisely when the overlay is broken.

  2. There is no stop / suspend / restart verb in the CLI or the public API (checked rc47 and rc48; docs confirm). A plain machine stop/start preserving the volume would very likely clear the wedge (it is VM-side mount state, not necessarily volume data), but there is no way to ask for one.

  3. DELETE /v1/sprites/{name}/exec/{id} returns 404 despite being documented (verified on rc47), so the zombie exec sessions that pile up from client retries cannot be cleared.

  4. The container cannot be stopped either: POST .../services/container/stop → "service not found" (the user container is a hooks-managed service, not in the services API).

  5. Idle-suspend never fires as an escape hatch. After quieting all clients, phantom http: activity sources persisted for 25+ minutes — server-side request handlers stuck on wedged reads (one hung checkpoints GET holds an activity source forever) plus pooled keep-alive connections — so the sprite never reaches the quiet window that would let it suspend and cold-boot.

  6. Killing the container's init hold-open from inside (08-24, when exec still worked) SIGKILLed the container tree and destroyed its mount namespace — and the supervisor never logged a container exit and never recycled the VM; its teardown appeared stuck unmounting the wedged overlay.

Ask

  • A supervisor- or platform-side hard recycle (stop + start with the volume intact), exposed via API/CLI, that does not depend on the overlay manager being healthy.
  • Request handlers that block on storage should time out and not count as idle-blocking activity, so a wedged sprite can at least drain to suspend.
  • Fix or un-document DELETE /v1/sprites/{name}/exec/{id}.
Dominant language
TypeScript
Stars
51
Forks
4
Avg merge
23h 57m
Merged PRs (30d)
12

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from superfly/sprites-docs

All issues in superfly/sprites-docs

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.