overengineeringstudio / overengineeringstudio/effect-utils
Epic: hermetic dependency artifacts for pure cross-worktree reuse
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 82
- Forks
- 2
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 121
Description
Problem
PR #922 fixes dependency-edge authority and retains major cross-worktree byte
reuse, but its current whole pnpm Store Cache is not the long-term purity
maximum. v11/index.db is shared writable SQLite state whose identity is the
ambient union of installs. It is disposable and non-authoritative, but it is
neither declared-input-addressed nor immutable.
The accepted dependency-materialization contract now makes the distinction
explicit:
- only declared-input-derived, integrity/content-addressed, immutable state is
eligible for cross-root reuse; - current GVS is inadmissible because it shares mutable topology/repair state;
- whole-Store-Cache index sharing is a transitional implementation delta;
- root-local mutable realization is a compatibility boundary, not the target.
The earlier split-v11/files design is not a solution: independent indexes did
not discover the shared data effectively (12 reused / 600 downloaded in the
real two-root experiment), and native profile-local prune deleted data required
by sibling roots.
Outcome
Make the reusable unit a Hermetic Dependency Artifact keyed by the complete
Materialization Profile and consumed read-only across every compatible root.
Only root-specific workspace projection, mutable lock-update work, and explicit
native leaves remain root-local.
Use an immutable, atomically published Store Cache seed with root-local writable
indexes/overlays only as a stepping stone when it directly reuses the existing
prepared-deps producer and converges on the same artifact identity. Do not
create a second cache/profile vocabulary or synthesize pnpm's private SQLite
schema.
Hard constraints
- Lifecycle scripts, ambient downloads during consumption, side-effect caches,
mutable native/build output, and consumer mutation are forbidden. - Identity includes lockfile, workspace manifests/members, catalogs, patches,
pnpm/Node versions, install policy, platform/optional/native classification,
and every other topology-affecting input. - Publication is atomic; consumers observe an old complete artifact or a new
complete artifact, never a partial one. - Cross-root reuse does not grant graph, mutation, repair, prune, or GC
authority. - Eviction/corruption of one root's writable state cannot affect another root or
the reusable artifact. - Linux hardlinks alone are not an immutability boundary: attempted writes
through every imported alias must fail or leave the artifact hash unchanged. - GC is artifact-owner reachability, not mutually invisible pnpm indexes over a
shared files pool.
Architecture
declared Materialization Profile
-> private lifecycle-free producer
-> validate + normalize + hash
-> atomic immutable publication
-> broad read-only reuse
-> root A workspace/projection state
-> root B workspace/projection state
-> CI/Nix consumers
Reuse the existing mk-pnpm-deps.nix prepared-dependency/FOD and profile model
as the starting producer. pnpm remains the resolver and lockfile writer; the
artifact path is a faithful materialization consumer, not a second resolver.
Required experiments and gates
- Identity completeness: perturb every semantic input; each relevant
perturbation changes the artifact key, while source-only edits do not. - Reproducibility: two clean private builds per supported system are
byte-identical; negative fixtures prove zero lifecycle execution. - Atomic publication: kill the producer at every phase; no consumer can
observe a partial artifact. Corrupt artifacts fail integrity validation. - Network denial: a second absent root consumes the artifact with transport
denied and records zero connections/bytes, not merely reporter counters. - Mutation isolation: root A install/update/prune/corrupt/repair leaves the
artifact and root B byte-identical. - Concurrency: two cold roots consume one artifact without a shared
writable DB or cross-root mutex; crash injection cannot poison siblings. - Physical reuse: report logical/allocated/unique bytes, files/inodes, and
shared-block ratio on Linux/ext4 and Darwin/APFS. - Latency/work: compare cold, second-root, warm, offline, repair, and
concurrent phases against #922's whole-store baseline and isolated stores. - Recovery/GC: independently evict/rebuild a root overlay and the artifact;
reclaim one artifact generation without coordinating unrelated consumers. - Live compatibility: prove workspace live-source links, lock mutation,
projection freshness, exact native-leaf replacement, and prepared CLI/Nix
consumers.
Benchmark baseline
Current #922 effect-utils/ext4 measurements (629 packages):
| realization | mean | second-root reuse | shared blocks |
|---|---|---|---|
| split files + clone/copy | 24.7015 s | 12 reused / 600 downloaded | 0.7% |
| root-local index + hardlink | 15.056 s | 12 reused / 600 downloaded | 89.6% |
| whole shared Store Cache | 6.8815 s | 612 reused / 0 downloaded | 89.6% |
The new design must satisfy purity/authority gates first, then seek a
non-dominated point among evaluated admissible candidates. A speed or byte win
cannot legalize shared mutation.
Delivery
- Record the current whole-store path only as the compatibility baseline and
keep DMP DELTA-001 open until code converges. - Prototype immutable seed + root-local writable overlay without production
adoption; falsify it if pnpm cannot consume it safely. - Measure direct live consumption of the existing prepared-dependency
artifact before inventing a new producer. - Select seed stepping stone or direct Hermetic Dependency Artifact through
the hard gates and same-workload benchmark matrix. - Implement one authoritative producer and root-local projection consumer.
- Add typed CI scenarios under #945 for identity, completeness, correctness,
reuse, storage, latency, concurrency, and recovery. - Prove effect-utils, dotfiles/Vista, and one simpler downstream consumer on
Linux and Darwin. - Remove whole-store shared-index leases/prune/migration behavior and delete
DMP DELTA-001 plus fleet DELTA-001. - Update PR #922 and downstream merge contracts so no artifact claims the
compatibility realization is the long-term architecture.
Related and ownership
- #922 — current compatibility implementation and measured baseline
- #945 — typed measurement scenarios and truthful completeness
- #353 — canonical workspace topology planner and shared identity inputs
- schickling/dotfiles#502 — earlier Nix-centric lockfile-consumer exploration;
this issue narrows ownership to effect-utils and integrates the live-worktree
path - schickling/dotfiles#1306 — fleet consumer and native projection
effect-utils owns the artifact/profile/materialization semantics. Consumers own
their workload declarations and root-specific projections; fleet owns placement
of immutable artifacts and local writable overlays, not dependency identity.
Posted on behalf of @schickling
| field | value |
|---|---|
agent_name |
🦜 co3-macaw |
agent_session_id |
2704af84-dabe-4279-99fb-e8e7c3920c7a |
agent_tool |
Codex CLI |
agent_tool_version |
0.144.1 |
agent_runtime |
Codex CLI 0.144.1 |
agent_model |
unknown |
runtime_profile |
/nix/store/qq3avrif77r90ypqbdv3hgd3gvwj5s32-coding-agent-runtime-profile/share/coding-agents/profile.json |
skills_manifest |
/nix/store/pjlb3cwf453ghzmc8jj4v8h29sw6p4dg-agent-skills-corpus/share/agent-skills/manifest.json |
worktree |
effect-utils/schickling-assistant/2026-07-17-pnpm-projection-authority |
machine |
dev3 |
tooling_profile |
dotfiles@4b8e1c5 |
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing mk-pnpm-deps.nix prepared-dependency/FOD and profile model, then review the compatibility baseline in #922 and typed scenarios planned under #945. Compare the proposed producer and root-local projection against the listed identity, atomicity, network, mutation, concurrency, reuse, and recovery gates. Done means one authoritative implementation passes the required scenarios for the named consumers on Linux and Darwin.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- build-system, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100