jeswr / jeswr/solid-server-rs

Handover: solid-server-rs β†’ SPARQ ownership + sparq-workspace sub-crates (core + lws + solid)

Open
#14 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

# Handover: `solid-server-rs` β†’ SPARQ ownership + sparq-workspace sub-crates

> πŸ€– **From the PSS agent** (@jeswr's agent for `prod-solid-server` / the Solid app+Pod-Manager
> suite) **to the SPARQ agent.** The maintainer has directed that `solid-server-rs` move under
> SPARQ's management and become sub-crate(s) of the sparq workspace. This is the context you should
> have **before** starting that work and taking on ownership. Nothing here is urgent β€” it's the map.

---

## 0. TL;DR / the one decision that shapes everything

Today `solid-server-rs` is a **single crate** that reaches SPARQ **over HTTP** (`HttpSparqClient`,
M3). The moment it lives inside the sparq Cargo workspace, that HTTP client should become an
**in-process crate dependency on the sparq engine** β€” WAC evaluation and access-controlled SPARQL
(the `sparq#992` integration) become **direct API calls into sparq's access-control graph engine**,
not a network round-trip. That single change is the whole reason the move is worth doing, and it
also unblocks WAC (see Β§4).

The maintainer's second instruction β€” **make it 3 crates (core + lws + solid)** β€” is correct and
retires the current `feat/lws` *branch* strategy (Β§3). You end up with one workspace, one shared
core, two thin protocol profiles, and no divergent server branch to keep merging.

---

## 1. What `solid-server-rs` is β€” and the guardrails (non-negotiable, maintainer-set)

- **EXPERIMENTAL, parallel-track.** It does **NOT** replace or override the live TypeScript
`prod-solid-server` (PSS). Maintainer directive 2026-06-20. Keep the "⚠️ EXPERIMENTAL β€” NOT a
production server" framing; **never** describe it as production-ready/live/supported in any public
setting.
- **SPARQ-authoritative.** SPARQ is the source of truth for RDF data, metadata, containment, and
(once WAC lands) access control. `object_store`/S3 is **backup/bytes-only**, never authoritative.
(This is the deliberate inverse of PSS's QLever-is-truth model β€” and per the standing directive,
**never QLever; use sparq**, building any missing capability *into* sparq.)
- **Auth is DPoP-bound Solid-OIDC, not bare Bearer.** Verification is issuer-agnostic.
- **Stack:** axum 0.8 / hyper 1.x / tokio, rustls + **aws-lc-rs** (FIPS-capable, the house crypto
backend), `axum-server` TLS, `oxttl`/`oxjsonld` for RDF, `object_store` for blobs.
- **Crypto backend is process-wide and load-bearing:** `main` installs the aws-lc-rs default
provider once; several components (TLS, the SSRF-guarded fetcher) assume it. Don't add a second
provider.

## 2. Current state β€” milestones, live branches, what's deferred

**Landed on `main`** (HEAD `1e555b1`):
- **M1** β€” boot + DPoP auth-middleware + LDP GET/HEAD/PUT over a `Store` trait (SparqClient +
BlobStore, in-memory test doubles) + LDP target parse + Turtle/JSON-LD via oxttl/oxjsonld.
- **M2** β€” POST / DELETE / N3-PATCH (`text/n3` insert/delete subset) / conditional (ETag, If-Match,
If-None-Match) / Range / conneg (Turtle↔JSON-LD); atomic `create_child`.
- **M3** β€” the live `HttpSparqClient` over the SPARQL 1.1 protocol with an **injection-safe
IRI/literal builder** + a mock-endpoint suite; replaces the in-memory `SparqClient` double.
- **WebID-outside-pod** (just landed, `8718582`β†’`c6d9f40`, +ETag/demoted-card fixes) β€” provider
WebIDs served OUTSIDE pod storage at an **id-host** (`id./#me`), a reserved
GET/HEAD-only **no-WAC** `/.identity` namespace, LDP refusal at two chokepoints, seed + owner-ACL
binding, design ADR `docs/design/webid-outside-pod.md`. Ported from PSS `decisions/0020`. **Keep
the id-host convention identical to PSS** — a future PSS→RSS migration must carry no identity
change.
- Production-hardening slices already in `main`: rate limiting, body-limit, overload/admission
(runs *before* auth/WAC/crypto β€” fail-safe), Redis replay store, TLS stateless-ticket resumption
(opt-in, **0-RTT off**), notifications hub (WebSocketChannel2023), security headers, CORS
(hand-rolled for the CTH's case-sensitive `Vary: Origin`).

**In-flight branches you inherit** (`origin/*`):
- `feat/lws`, `feat/lws-conformance-gaps`, `feat/lws-step8-ab` β€” **the LWS work as a divergent
branch. This is what the 3-crate refactor replaces** (Β§3). Read these first; they encode the LWS
intent but should be re-homed into the `lws` crate, not merged as a branch.
- `perf/write-coalescing-nl48`, `perf-a-http2-guards`, `perf-b-mimalloc`, `perf-c-alloc-reduction`,
`phase-existence-non-disclosure`, `phase-3-post-acl-guard-verify` β€” the throughput/perf backlog
(see Β§6, and the EC2 bench-box bead `za9r`).
- `conformance/protocol-fixes`, `conformance/wire-cth-baseline` β€” Solid CTH conformance (target
41/41).
- `chore/repin-async-dns-verifier`, `fix/unique-blob-keys` β€” housekeeping.

**Deferred (marked `// M2-next:` in code):** full **WAC evaluation (gated on `sparq#992`)**, the
N3-PATCH `WHERE` solver, notification reconciler/GC, and hardening beyond the current slices.

## 3. The 3-crate refactor (the maintainer's structural ask) β€” recommended shape

Split the single crate into a workspace with **three crates**, so LWS and Solid are *profiles over a
shared core* rather than a forked branch:

- **`lws-core`** (protocol-agnostic substrate) β€” the `Store` trait + its **in-process sparq
binding** (replacing `HttpSparqClient`), the `object_store` blob backend, the axum/LDP HTTP
plumbing (verb handlers, conditional/Range/conneg, the N3-PATCH engine), RDF parse/serialise
(oxttl/oxjsonld), the injection-safe SPARQL builder, target parsing, `create_child`, the
DPoP/PoP auth-middleware **seam**, overload/rate-limit/body-limit, TLS, notifications transport.
Everything both protocols share.
- **`lws`** (the LWS profile) β€” the LWS document **conformance classes**, the access-decision
semantics, LWS discovery / storage-description / notification surface, **RDF-opt-in** per the LWS
direction. Its contract is the `jeswr/lws-spec` **test-vectors + executable oracle** (Β§5.1) β€” the
crate must reproduce them.
- **`solid`** (the Solid Protocol profile) β€” Solid-OIDC (via `solid-oidc-verifier`, Β§5.4),
WAC/ACP, the id-host WebID surface, type-index conventions, the Solid CTH conformance surface.

Notes:
- **The `Store` trait is the seam that makes this clean** β€” its sparq implementation moves from an
HTTP client to a direct crate call on sparq's engine. Keep the trait; swap the impl. That's the
single highest-leverage change and it's where WAC (Β§4) plugs in.
- `solid-oidc-verifier` (Β§5.4) is already its own crate β€” depend on it, don't absorb it (it's
independently useful + security-audited; ES512 behind a feature).
- Binaries: a `solid-server` bin (core+solid) and optionally an `lws-server` bin (core+lws), or one
bin selecting the profile by config. Your call.

## 4. WAC is gated on `sparq#992` (AC-SPARQL) β€” and you own both sides now

The single biggest deferred item is **WAC authorization**, deliberately gated on **`sparq#992`**
(the Access-Controlled SPARQL design β€” I wrote the ReSpec Editor's Draft + mirrored it to that
issue; the sparq side is yours). The design intent (`README` + the WAC seam in code): **ACL lives in
the SPARQ access-control graph = source of truth, and SPARQ does the efficient per-resource WAC
decision** β€” the server asks sparq "may agent A do mode M on resource R?" rather than fetching ACLs
and evaluating in Rust. As an in-process sub-crate, that becomes a direct engine call. Owning both
`sparq#992`'s implementation *and* the server that consumes it is the reason this consolidation is a
win β€” you can co-design the API instead of negotiating it across an HTTP boundary.

**⚠️ Known HIGH finding to carry across:** the **POST-Slug `.acl` WAC bypass** β€” a `POST` that mints
a child named `.acl` (or otherwise lands on the ACL-document chokepoint) can be a
privilege-escalation vector; it's live in **both** `solid-server-rs` and PSS, and was found by
adversarial verify (CTH + roborev missed it). Fix = guard the POST child-name chokepoint **before**
WAC lands, and make sure the new WAC path closes it. (`phase-3-post-acl-guard-verify` is the branch
addressing this side.)

## 5. Specs I created/proposed that you should know (the LWS + auth + conformance stack)

### 5.1 `jeswr/lws-spec` β€” the clean-slate LWS spec + **executable conformance** (the `lws` crate's contract)
The most important one. A ground-up ("JLWS") LWS protocol spec with a **machine-readable
statement-companion**, **~157 language-neutral test-vectors**, and an **executable N3 oracle** (run
by EYE / `eyereasoner`) that re-derives the expected access decisions. Includes:
- Per-**document conformance class** normative **SHACL shapes**.
- A normative **N3 access-decision rule set** as the definitional oracle (the `lws` crate must
reproduce its verdicts).
- **TLA+ temporal models** β€” which caught + fixed a real **revocation two-clocks oracle-window** bug
(resolved to single-clock / acknowledgement-as-barrier).
- **ODRL prohibition/obligation decision-time composition** (deny-overrides + obligation
fail-closed, per-grant + matching-scoped; duty-discharge deferred as `D23`).
Treat the vectors + oracle as the conformance bar for the `lws` crate: an independent implementation
is *conformant* iff it reproduces them. (This is deliberate β€” the "measurable second implementation"
exit criterion; your Rust `lws` crate is a candidate second implementation.)

### 5.2 `jeswr/lws-ucs` β€” Use-Cases & Requirements + the `lws-ucr` ontology
83 use cases / 73 requirements expressed with a formal OSLC-RM-based ontology, a machine-checkable
chain UC β†’ requirement β†’ SHACL/N3 β†’ test-vector β†’ EARL, all 137 upstream issues triaged, plus
21 app-derived UCs (offline-first, CRDT logs, atomic hydrate, etc.). Context for *why* the LWS
surface is shaped as it is.

### 5.3 `jeswr/dpop-sk-spec` β€” DPoP-SK (high-throughput auth Tier-2)
A Solid-OIDC PoP-negotiation profile: one DPoP proof at session establishment β†’ an HKDF-derived
symmetric session key β†’ per-request RFC 9421 HMAC attestation (no per-request asymmetric verify).
DPoP stays the mandatory baseline (Tier-1). This feeds `solid-server-rs`'s **Tier-2 fast-path** β€”
see the `pop/` module + `docs/design/high-throughput-pop-auth.md`. The review found + fixed a real
RFC-9266 channel-binding flaw. Relevant when you push throughput (Β§6).

### 5.4 `jeswr/solid-oidc-verifier` β€” the Rust DPoP-bound token verifier (already a dependency)
A standalone crate: RFC 9449 DPoP + RFC 9068 `at+jwt` + issuer-agnostic + RFC 7638 thumbprint +
DNS-pinned SSRF WebID resolver + ES512 (feature-gated). Security-hardened over 13+ adversarial
rounds. `solid-server-rs` depends on it (git-dep, pinned rev). **Recommendation: keep it standalone**
in the workspace as a path/git dep, not absorbed. Its `JwksProvider`/`ReplayStore`/`WebIdResolver`
seams are the extension points.

### 5.5 WebID-outside-pod β€” landed in **both** servers
`decisions/0020` (PSS) + `docs/design/webid-outside-pod.md` (RSS). Provider WebIDs live outside the
LDP-mapped storage so WAC can never apply to them (prevents accidental grants over OIDC statements).
Keep RSS's id-host convention byte-identical to PSS's.

### 5.6 Other context
`jeswr/agentic-solid-conformance` (shared golden conformance vectors), and the throughput specs in
`docs/design/` (`beyond-50k-throughput.md`, `throughput-hard-cases.md`). The maintainer's LWS
directive (2026-07-05) explicitly says: **throughput hard-cases (many queries / large pods / complex
ACL) β†’ file sparq issues** β€” those are now *internal* to your workspace.

## 6. Throughput / perf is a live theme, and it converges on sparq
There's a substantial perf backlog (the `perf/*` + `phase-*` branches, the `beyond-50k` + hard-cases
design docs, the EC2 bench-box phases in bead `za9r`: syscall baseline, vectored-write audit,
SO_REUSEPORT, kTLS, io_uring/monoio). The recurring bottleneck is the SPARQ round-trip β€” which the
in-process sub-crate move directly attacks. CTH must stay green (41/41) through any perf change.

## 7. Cross-agent / process notes
- **Ownership transfer:** it's currently mine (PSS agent). After the move it's yours. I'll stop
treating it as a PSS repo; ping me if you want the WebID-outside-pod / id-host details or the PSS
`decisions/0020` rationale β€” the two servers must keep identical identity conventions.
- **Review gate:** every commit is auto-reviewed by **roborev / codex `gpt-5.6-sol`** (post-commit
hook + `.roborev.toml`); the global config resolves the model. Keep that on the new crates.
- **Tracker:** cross-repo work lives in the private `jeswr/suite-tracker` beads store; per-repo
user-visible items are GitHub issues. If you point your `bd` at the same store it becomes the one
shared cross-agent board (the maintainer floated this).
- **`TAG NO ONE`** on the upstream LWS/W3C side (maintainer directive) β€” the LWS work is prepared
quietly; the maintainer drives any outreach.
- **The specs are the contract, not my code** β€” where the Rust crates and the spec/oracle disagree,
the spec/oracle wins (or you propose a spec change on `lws-spec`, which I can review).

## 8. Suggested first moves (non-binding)
1. Read `feat/lws` + `feat/lws-conformance-gaps` to see the current LWS intent.
2. Stand up the workspace skeleton: `lws-core` + `lws` + `solid`, move the single crate's modules
into `lws-core`, keep `main`/CTH green at each step.
3. Replace `HttpSparqClient` with the in-process sparq-engine binding behind the `Store` trait.
4. Land WAC through the sparq access-control engine (`sparq#992`), closing the POST-Slug `.acl`
bypass in the same pass.
5. Wire the `lws` crate to the `lws-spec` vectors + oracle as its CI conformance gate.

---
*Questions β†’ reply here or open an issue and tag the PSS agent. β€” πŸ€– PSS agent*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the current Store trait and HttpSparqClient, then compare the feat/lws and feat/lws-conformance-gaps branches with the proposed three-crate layout. Preserve the existing experimental-server guardrails and WebID design in docs/design/webid-outside-pod.md. Done means a workspace with lws-core, lws, and solid crates, an in-process SPARQ binding, and profile behavior aligned with the stated LWS and Solid conformance surfaces.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, build-system
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.