overengineeringstudio / overengineeringstudio/effect-utils
mkPnpmCli: eliminate IFD to support cross-system flake evaluation
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 82
- Forks
- 2
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 121
Description
Problem
mk-pnpm-cli.nix uses Import From Derivation (IFD) — it calls builtins.readFile and lib.cleanSourceWith on the workspaceRoot argument during Nix evaluation. When workspaceRoot is a derivation (e.g. from pkgs.runCommand), this forces the derivation to be built at eval time.
This breaks cross-system evaluation. For example, when an aarch64-darwin machine evaluates x86_64-linux outputs (via eachDefaultSystem), it can't build the system-specific runCommand derivation locally. nix flake check and nix eval fail without remote builders.
IFD locations in mk-pnpm-cli.nix
- Line 74:
builtins.readFile (workspaceRootPath + "/${packageDir}/pnpm-workspace.yaml") - Line 306:
builtins.fromJSON (builtins.readFile (workspaceRootPath + "/${packageDir}/package.json")) - Lines 177-178, 275-278:
lib.cleanSourceWith { src = workspaceRootPath; ... }
Why consumers use derivations as workspaceRoot
When a CLI package lives in a different repo structure than its workspace dependencies (e.g. a dotfiles repo consuming @overeng/* packages from effect-utils), consumers build a combinedWorkspace derivation that:
- Mounts the main package and external deps into a unified directory tree
- Rewrites
pnpm-workspace.yamlto match the new layout - Patches
pnpm-lock.yamlimporters/specifiers with sed/yq (fragile) - Patches
package.jsonto fix paths
This is 50-100 lines of brittle runCommand boilerplate per consumer (6 consumers in dotfiles alone).
Proposed solution: internalize workspace assembly
Replace the current API where consumers pre-build a workspace derivation with a new API where mkPnpmCli accepts individual source ingredients and assembles the workspace itself at build time (not eval time).
Key insight
Mount workspace deps at positions matching the lockfile's relative paths. The lockfile stays untouched — eliminating all sed/yq rewriting.
Current API
mkPnpmCli {
name = "otel";
workspaceRoot = combinedWorkspace; # ← derivation, triggers IFD
packageDir = "flakes/otel-cli";
entry = "flakes/otel-cli/bin/otel.ts";
pnpmDepsHash = "sha256-...";
}
Proposed API
mkPnpmCli {
name = "otel";
packageSource = ./.; # plain Nix path — no IFD
entry = "bin/otel.ts"; # relative to package root
# Workspace members: mounted at lockfile-relative positions
workspaceDeps = {
"../tui-core" = effectUtilsSrc + "/packages/@overeng/tui-core";
"../tui-react" = effectUtilsSrc + "/packages/@overeng/tui-react";
"../utils" = effectUtilsSrc + "/packages/@overeng/utils";
"../utils-dev" = effectUtilsSrc + "/packages/@overeng/utils-dev";
};
# Deps mounted at path but NOT added to workspace yaml
linkDeps = {};
# jq filter applied to package.json at build time
packageJsonFilter = ''del(."$genie")'';
patchesDir = "../utils/patches";
pnpmDepsHash = "sha256-...";
}
Consumer patterns identified
| Pattern | Consumers | workspaceDeps | linkDeps |
|---|---|---|---|
| Simple (no external deps) | bird, gh-ci-exporter | {} |
{} |
| Link dep only | op-secret-cache, agent-exporter | {} |
{ "../utils" = ... } |
| Workspace members | otel-cli | 4 members | {} |
| Complex hybrid | oi | ~10 members | { geist = ... } |
Benefits
- No IFD: All eval-time reads (
builtins.readFile,cleanSourceWith) operate on plain paths/flake inputs - No lockfile patching: Mounting at lockfile-relative paths means zero sed/yq rewriting
- Less boilerplate: ~80 lines of
combinedWorkspaceper consumer → ~10 lines ofworkspaceDeps - Cross-system eval:
nix flake checkworks without remote builders
Open questions
- Should
packageJsonFilterbe a jq expression string, or a Nix function? - How to handle oi's
catalog:patching in livestore deps —prePatchscript param vsoverrideAttrs? - Should the builder preserve non-
packagesfields from the originalpnpm-workspace.yaml(e.g.supportedArchitectures,publicHoistPattern)?
Context: dotfiles#254
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with mk-pnpm-cli.nix, especially the IFD locations at lines 74, 177-178, 275-278, and 306, then review the listed consumer patterns and the dotfiles#254 context. Use nix flake check and nix eval to verify cross-system evaluation. Done means workspace assembly occurs at build time, the lockfile is not patched, and the proposed API handles the documented consumer cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- build-system
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100