CUBE gap: a directory has no archive representation, so every client orchestrates a feed to download one
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 0
- Avg merge
- 1h 35m
- Merged PRs (30d)
- 333
Description
Standing CUBE gap, recorded in `docs/CUBE-gaps.adoc` under "Confirmed proposal: directory archive".
## The requirement
Take a copy of a directory. A feed's output, a series pulled from PACS, a plugin's working set — getting a tree of files out of CUBE and onto the machine a person is sitting at.
A single file is straightforward: CUBE serves its bytes. A **directory has no representation CUBE can hand over**, and that absence is the gap.
## The burden today
Every client builds the archive out of the platform's own parts. The established workaround — arrived at independently in the ChRIS web UI years before mise — is:
1. create a feed
2. attach `pl-dircopy` rooted at the directory
3. attach a zip plugin downstream
4. poll until the job completes
5. serve the resulting archive as an ordinary file
Using a plugin for the archiving itself is **correct**. "Make an archive" is a computation, and computations here are plugins; inventing a separate zip service would be worse. The problem is everything around it.
**The orchestration has no home.** Feed creation, plugin attachment and completion polling are a workflow, and CUBE offers no way to express it as one thing. Every client re-implements it, and the sequence exists only as REST calls inside somebody's source. In the web UI this is exactly how a presentation layer became an orchestrator — business logic and rendering co-mingled in a client no other client could reuse. mise confines it to the intent kernel so every surface reaches one implementation, but it is still a workflow standing in for a missing route.
**It pollutes provenance.** A download creates a feed. The compute DAG acquires an entry asserting that someone took a copy, as though that were an analysis step. Feed lists fill with archive jobs, and the graph that exists to answer "how was this derived" answers a question nobody asked.
**It is slow and failure-prone out of proportion.** Copying a directory should not require scheduling compute, waiting on a job queue, or recovering from a plugin that failed halfway. The latency is that of a pipeline run rather than a read.
**It leaves an artifact nobody asked for.** The archive persists in CUBE storage afterwards, and no client owns cleaning it up.
## Proposed interface
An archive representation of an existing directory, through the ordinary collection graph rather than by scheduling compute:
```
GET /api/v1/filebrowser//archive/?format=zip
```
The exact shape is CUBE's to choose — a link relation on the existing filebrowser collection item is the natural seam, since that collection already represents a directory, knows its contents, and knows the caller's permission to see them. What matters is that a directory acquires a fetchable representation.
Required semantics:
- applies CUBE's normal authorization; never includes a file the caller could not fetch individually
- **streams**, so a large tree does not require materializing the whole archive before responding
- creates no feed, no plugin instance, no durable artifact — a read leaves the compute graph untouched
- bounds or reports its size, so a client can refuse or warn before starting
- names its format explicitly rather than implying one
## Why an existing collection is insufficient
The filebrowser collection enumerates a directory's contents, so a client can fetch every file and assemble an archive locally. That is an N+1 traversal, with the added cost that a browser must hold the whole tree in memory to build a zip it then hands to the download manager.
The plugin route is not an interface. It is a client compensating for a missing one, and it is filed because **it works well enough to be invisible** — the condition under which a gap stops being felt and never gets fixed.
## Client policy until then
mise walks the directory and fetches each file individually — `files_listRecursive` plus one request per file. That needs no compute and leaves the compute graph clean, and it is acceptable only because a local shell writes the files as they arrive.
It does not survive a surface with no filesystem: a browser cannot receive several hundred files as several hundred saves. So mise adopts the archive workflow — `pl-dircopy` into a zip plugin — for exactly that case, in `brasa` rather than in a surface, so every surface reaches one implementation and no client re-derives the sequence. The archive is a CUBE artifact with an ordinary path, delivered through the surface capability like any other file (#232); no daemon filesystem is involved. The workaround announces itself: a directory download reports that it is creating a feed in order to do so.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with docs/CUBE-gaps.adoc and the directory-download path that uses files_listRecursive and the brasa archive workflow. Review how issue #232 delivers ordinary files through the surface capability. Done means an agreed directory archive interface with authorization, streaming, explicit format and size behavior, without creating feeds, plugin instances or durable artifacts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100