posit-dev / posit-dev/hephaestus

server/client mode

Open
#15 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
20
Forks
1
Avg merge
9h 18m
Merged PRs (30d)
4

Description

A start-basis from the work on a plot representation format for #14 when it comes to borrowing the codec macros for object transfer:

What transfers unchanged

codec.rs is a generic hand-rolled binary serializer — Writer/Reader, LEB128 varints, zigzag signed, the blanket impls for Option/Vec/Box/arrays/HashMap, and impl_codec! with its four struct/enum/newtype/generic forms. Nothing in it knows about plots. The ~2500 lines of impls_*.rs are the real asset: every Theme, Scale, Geom, style-vocabulary type already round-trips, and a command like "replace this scale" or "update this geom's columns" is mostly a wrapper around impls that already exist.

Two properties make it a better starting point than a generic serde codec for a network boundary:

Decoding goes through builders, so a peer sending non-increasing bin edges gets DocumentError::Invalid rather than a scale that misplaces rows (codec.rs conventions, impls_scale.rs).
Reader::count bounds every length by remaining bytes before Vec::with_capacity (codec.rs:525 and the count helper), so a hostile length prefix can't induce an allocation larger than the frame. That's the classic bug in hand-rolled wire formats and it's already closed.
What doesn't transfer

  1. wire.rs is a file container, not a frame format. Magic + version + u32-length chunks per message means 16 bytes of preamble on every frame plus 8 per chunk, and version negotiation belongs in the WS handshake, not in each message. You'd want a sibling to wire.rs: one varint message tag, no magic, version agreed once per connection.

  2. Interning is per-document and that's the hard part. Writer owns WriteTables and finish() consumes it (codec.rs:41-95). A session wants the opposite: send a geometry once at frame 1, reference index 7 for the next thousand frames. The detached() trick shows the tables are structurally movable, so the refactor is "borrow a session-owned table" rather than "own one" — but the semantics change materially:

The dictionary becomes protocol state both sides must agree on. WS is ordered and reliable per connection, so that works, but reconnect must reset it.
A session table holds every Arc forever → unbounded growth on a long streaming session with changing data.
If you add eviction, HashMap<*const Geometry, u32> becomes unsound. It's safe today only because the table keeps the Arcs alive for the write's duration, so an address can't be recycled. Evict, and a freed address reused by a new Geometry silently maps to the wrong index — the client draws the wrong shape with no error. Design around it with a generation counter, content hashing, or eviction only at explicit epoch boundaries.
3. Identity is deliberately dropped, and a command protocol needs it. The "Known limitation" in src/document/CLAUDE.md — GeomId/AxisId/LegendId aren't carried, replay renumbers from zero — is exactly backwards for commands: update_geom(GeomId, …) is the thing that has to survive the wire. Handles do need to be carried, and both sides need to agree on the issuing authority (server-assigned, client echoes).

  1. The feature split doesn't map. document-read/document-write assume one direction per build. A server writes commands and reads events; the client does the mirror. Both need both halves. Since the codec's #[cfg]s name the document-* features literally, you'd want internal codec-read/codec-write gates that both document-* and a new protocol feature imply — otherwise a websocket build pulls in the whole document write path to get a Writer. The upside: the no-renderer 1.86 configuration already proves a server that never rasterizes can build without wgpu.

  2. Shared discriminant space becomes doubly frozen. "Renumbering silently reinterprets every document already written" would extend to "…and desyncs every peer on an older build". Fine, but keep command-enum numbering in its own module rather than extending the document's; the value impls are shared and therefore constrained by both.

The framing question underneath

There are two protocols you could build, and only one is 90% written:

Thin protocol / thick client — ship the composition, then config deltas; the client re-solves layout and shapes text at its own size. This is the document codec, plus deltas. Also gets you resize for free: it's already the format's premise.
Thick server / thin client — solve on the server, ship scene primitives per frame. Needs codec impls for src/primitives/ and the SceneBuilder op stream, which don't exist, and every frame carries shaped glyphs and solved geometry — which the document format exists specifically to avoid writing.
Go with the first. The one place a document decision is arguably wrong for the wire is unquantized f64/f32 — deliberate and correct for scale domains, expensive at 8 bytes/coordinate for streaming point data. Quantization or delta coding of channel columns would be an addition to the codec, not a rework of it.

Net: extract codec.rs + intern.rs + the impls_*.rs value layer as a shared serialization core with session-scoped rather than message-scoped interning; write a new frame container beside wire.rs; and add the id-carrying command/event vocabulary that documents deliberately omit.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading codec.rs, intern.rs, the impls_*.rs value layer, and wire.rs, then review the identity and feature-gate notes in src/document/CLAUDE.md. Done means a shared serialization core, a sibling frame container, session-scoped interning, and an id-carrying command/event vocabulary suitable for both server and client modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.