get2knowio / get2knowio/deacon
auto-forward: persistent in-container multiplexing relay agent (drop per-connection docker exec)
- Dominant language
- Rust
- Stars
- 2
- Forks
- 0
- Avg merge
- 2h 36m
- Merged PRs (30d)
- 78
Description
Tracked deferral from `up --auto-forward` (spec `015-auto-forward-ports`, task **T055**; shipped in #188). A throughput optimization for the relay.
## Goal / acceptance
Relay no longer pays a `docker exec` per connection; same external behavior.
## Background — what v1 does
`crates/core/src/port_forward/relay.rs` spawns a **fresh `docker exec -i ` per accepted host connection**. Correct and dependency-light, but every connection pays docker-CLI fork + daemon round-trip + `setns()` + relay-program fork (~tens of ms + real CPU). Bad under high connection churn (many short HTTP requests, etc.).
## Proposed model
One long-lived `docker exec` carrying many logical streams over its single stdin/stdout, with a tiny framing protocol:
```
daemon ⇄ ONE `docker exec -i ` (started once per container)
OPEN | DATA | CLOSE
```
Host side allocates a `stream_id` per connection and demuxes replies; the in-container agent dials `127.0.0.1:` per `OPEN` and frames bytes back.
## Hidden issues / gotchas
- **Getting the agent into the container is the real blocker.** v1 relies on tools already present (`socat`/`nc`/bash). A multiplexer needs a program in the container that speaks the protocol:
- **Embedded static binary** (musl, few KB, `docker cp`'d in) — works on `alpine`/`distroless`, but means **building, arch-matching (amd64/arm64), versioning, and embedding a second artifact** in the deacon distribution + a cp/exec/integrity-check dance. (`research.md` Decision 3 named this the primary strategy and left the exact form an open sub-decision.)
- **Shell-based agent** — no new artifact, but fragile/slow and absent on `distroless`.
- **You now own a protocol:** stream IDs, **flow control / backpressure** (one slow stream must not head-of-line-block others on the shared pipe), half-close semantics, agent crash/restart, clean teardown.
- **Must preserve the no-silent-fallback guarantee:** if the agent can't be installed, fall back to today's per-connection relay (still works on any image).
## Recommended path forward
1. **Phase 1 (verifiable here):** implement the host-side framing/demux + a **shell-based** in-container agent; route `start_forward` through the multiplexer when available, else fall back to per-connection. Add Docker integration tests (concurrent streams, slow-stream isolation, agent death → fallback).
2. **Phase 2:** add the **embedded static musl agent** (cross-compiled, arch-matched, version-checked, `docker cp`'d) so it works on minimal images; wire it into the release build. This is the part that can't be fully exercised in the current sandbox.
3. Keep per-connection relay as the permanent fallback.
## Notes
Largest of the auto-forward deferrals (a protocol + a shipped artifact, not a contained feature). Relates to #189-style "ship an in-container agent" concerns shared with event-driven detection.
Contributor guide
Research direction
Start with crates/core/src/port_forward/relay.rs and the start_forward path, then read research.md Decision 3. Define the host framing and demultiplexing behavior, shell-agent fallback, and Docker integration coverage for concurrent streams, slow-stream isolation, and agent death. Done means the multiplexer preserves external behavior while retaining the existing per-connection relay fallback.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, rust
- Domain
- infrastructure, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100