roc-lang / roc-lang/basic-webserver
Add bounded streaming multipart staging and upload sinks
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 107
- Forks
- 20
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 11
Description
Summary
Build the multipart follow-up explicitly deferred by #185.
The host should stream multipart/form-data, capture small declared fields in bounded memory, and stage declared file fields under a writable-root capability. Roc receives metadata and opaque staged-file handles rather than every file byte.
Illustrative shape:
upload = request.body().stage_multipart!({
root: upload_root,
allowed_file_fields: ["avatar", "attachment"],
allowed_value_fields: ["caption"],
limits: {
max_parts: 8,
max_files: 2,
max_file_bytes: 10 * 1024 * 1024,
max_part_header_bytes: 8 * 1024,
max_field_bytes: 16 * 1024,
max_total_field_bytes: 32 * 1024,
},
})?
upload.files.get(0)?.publish!({
relative: destination,
mode: CreateNew,
})?
Exact types should be designed during implementation.
Why this belongs in the host
A streaming boundary parser sits directly between an untrusted HTTP transport and host files. Keeping the transfer in the host preserves backpressure, bounds memory, enables secure staging, and prevents every file byte crossing into Roc.
Unlike arbitrary callbacks, the application supplies a finite declarative plan before the transfer. This follows design.md's request-body sink model.
Required contract
- Require a valid
multipart/form-datacontent type and boundary. - Enforce independent limits for total body bytes, parts, files, per-file bytes, part headers, individual value fields, and aggregate in-memory fields.
- Preserve field order and duplicate names.
- Return text/value fields as bytes unless their character decoding is unambiguous.
- Treat client filenames and claimed content types as untrusted metadata only.
- Never use a client filename as a filesystem path.
- Stage files securely and exclusively on the selected writable root's filesystem.
- Return opaque staged handles whose final release removes unpublished files.
- Publish only to validated relative paths with the existing
CreateNewsemantics. - On malformed input, disconnect, timeout, shutdown, saturation, or limit failure, remove every staged file.
- Do not promise atomic publication across multiple files. Each publication result is explicit.
- Reject nested multipart in the first version.
- Track active multipart sinks, staged files, and high-water resource use.
Staging before publication permits the application to inspect bounded form fields and metadata before selecting final destinations without installing a callback into Roc.
Tests
- Boundary fragments split across every possible transport chunk position.
- Empty, quoted, and maximum-length boundaries.
- Repeated fields and repeated file names.
- Header-count/byte, part-count, file-count, and size limits.
- Malformed closing boundaries and truncated bodies.
- Disconnect, timeout, and shutdown at each stage.
- Cleanup when handles are dropped or publication fails.
- Traversal and client-filename attacks.
- Parser fuzzing and a corpus of real browser multipart bodies.
- Zero leaked files, permits, buffers, or handles.
Non-goals
- MIME-type trust or file-signature validation
- Antivirus, sandboxing, or content disarm
- Arbitrary application callbacks while parsing
- Client-selected filesystem destinations
- Unbounded text-field materialization
- Atomic multi-file transactions
- Background or resumable uploads
Acceptance criteria
- Declared multipart files stream to secure staging without materializing file bodies in Roc.
- Every parser and storage dimension has a finite limit.
- Unpublished handles clean themselves up on all paths.
- Publication reuses writable-root and safe-relative-path guarantees.
- The parser has cross-platform, fragmentation, and fuzz coverage.
References
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with design.md and the raw request-body sink issue #185, then trace the request.body().stage_multipart! and publish! entry points described here. Compare the proposed contract with the existing writable-root and CreateNew guarantees. Done means bounded streaming, secure cleanup, explicit publication results, and the listed fragmentation, failure, cross-platform, and fuzz coverage.
Written by the indexing model from the issue text.
Assessment
- Domain
- api, backend, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100