overengineeringstudio / overengineeringstudio/effect-utils
Implement first-class package-closure-scoped pnpm installs in shared devenv pnpm task
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 82
- Forks
- 2
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 121
Description
Motivation
PR schickling/schickling.dev#113 exposed a structural problem in the shared taskModules.pnpm task: consumers can list only the effect-utils packages they need, but the task still runs a workspace-root pnpm install in repos/effect-utils.
That means a downstream repo can be blocked by unrelated workspace members. In #113, schickling.dev needed packages such as:
@overeng/effect-rpc-tanstack@overeng/notion-effect-client@overeng/notion-effect-schema@overeng/react-inspector@overeng/utils
but the install also traversed @overeng/pty-effect -> @myobie/pty -> node-pty, whose lifecycle build required Python on the runner. That dependency was unrelated to the package closure schickling.dev needed.
The immediate schickling.dev fix used PNPM_CONFIG_IGNORE_SCRIPTS=true / npm_config_ignore_scripts=true in the effect-utils install task. That was intentionally a local unblocker, not the long-term design: install scripts should be avoided by package-closure scoping, not globally disabled as a way to hide unrelated lifecycle work.
Reproduction
From schickling.dev PR #113 context:
- Consume effect-utils through the megarepo checkout.
- Configure a nested effect-utils pnpm task with selected packages only.
- Run the downstream quality gate, e.g.
CI=1 devenv tasks run check:allor the nestedpnpm:install:effect-utilstask. - Observe that the task runs a whole
repos/effect-utilsworkspace install. - The install reaches unrelated
packages/@overeng/pty-effect, then@myobie/pty, thennode-pty, and fails when the lifecycle build cannot find the expected Python toolchain.
Current local workaround in schickling.dev:
(taskModules.pnpm {
workspaceRoot = "repos/effect-utils";
taskSuffix = "effect-utils";
preInstall = ''
export PNPM_CONFIG_IGNORE_SCRIPTS=true
export npm_config_ignore_scripts=true
'';
packages = [
"packages/@overeng/effect-rpc-tanstack"
"packages/@overeng/notion-effect-client"
"packages/@overeng/notion-effect-schema"
"packages/@overeng/react-inspector"
"packages/@overeng/utils"
];
})
Exploration Results
- Adding Python would make this one lifecycle script pass, but it encodes an unrelated package's build requirements into every downstream consumer of effect-utils.
- Passing
--ignore-scriptswas rejected by the shared pnpm task's purity guard, correctly: repo-local install flags should not be able to weaken the shared install contract. - Exporting
PNPM_CONFIG_IGNORE_SCRIPTS=trueinpreInstallunblocked #113, but it is an escape hatch around the guard and should not become the pattern. - The package list already exists at the devenv module boundary, and Genie already has package-closure concepts (
projectPnpmPackageClosure(...)/ workspace metadata). The missing piece is making the shared pnpm task install that closure as a first-class mode. - E2E exploration verified that
pnpm --filter <package-name>... installincludes dependency closure, while leading ellipsis is the opposite direction and includes dependents. - A temp effect-utils manifest/lock copy with filtered frozen install selected 6 of 28 workspace projects for the schickling.dev closure. It did not run
@myobie/pty/node-ptylifecycle scripts. - A temp fixture showed filtered install skipped an unrelated failing postinstall script, while full install failed.
- Important pitfall: with
enableGlobalVirtualStore: true, pnpm may still materialize unrelated lockfile packages under GVS links even if they are not linked into selected packages and their lifecycle scripts do not run. Scoped install narrows lifecycle blast radius; it is not a guarantee that pnpm never touches unrelated store projections.
Design Requirements
Implement an explicit package-closure-scoped install mode in nix/devenv-modules/tasks/shared/pnpm.nix.
The mode should:
- Accept selected workspace package paths or package names and derive the full internal dependency closure.
- Install/link only that closure, not every package in the workspace.
- Preserve the repo-root
pnpm-lock.yamlas the authoritative lockfile. - Preserve shared task purity defaults: frozen lockfile in CI, controlled store dir, clone-or-copy import method, side-effects cache disabled, strict store content checks, and no repo-side
--ignore-scriptsescape hatch. - Keep lifecycle scripts enabled for packages in the selected closure.
- Avoid lifecycle scripts for packages outside the selected closure by not installing them.
- Keep status hashing accurate: include the selected closure, relevant manifests, lockfile, pnpm workspace config, install scope, filter list,
preInstall, and projection state. - Keep
pnpm:cleanscoped to node_modules projections owned by the task. - Produce clear diagnostics that report workspace root, selected package roots, computed closure, filters, and pnpm log location on failure.
- Work for nested
workspaceRootvalues such asrepos/effect-utils. - Continue supporting full-workspace install as the default behavior.
Potential API shape:
taskModules.pnpm {
workspaceRoot = "repos/effect-utils";
taskSuffix = "effect-utils";
installScope = "packageClosure";
packages = [
"packages/@overeng/effect-rpc-tanstack"
"packages/@overeng/notion-effect-client"
"packages/@overeng/notion-effect-schema"
"packages/@overeng/react-inspector"
"packages/@overeng/utils"
];
}
Open design point: whether packages = [...] should change behavior directly, or whether a new explicit option such as installScope = "workspace" | "packageClosure" is safer. Prefer explicitness to avoid silently changing current repos.
Pitfalls / Tradeoffs
pnpm --filtersemantics are subtle. The implementation must use--filter "${packageName}..."to include dependencies and must not accidentally use the dependent direction.- Hoisted bins and root-level tooling can mask missing closure members. Tests should prove package-local commands resolve without relying on unrelated root workspace packages.
- The task currently validates node_modules projection health across root plus package paths. Scoped mode needs equivalent health checks that do not require unrelated packages to project.
- Root
node_modulesmay still be needed for pnpm workspace metadata. Avoid treating a minimal root projection as permission to install every workspace member. dependenciesMeta.*.injectedhandling must stay correct; injected source paths currently influence hashing.allowBuilds/ignoredBuiltDependencies/ignoreDepScriptsinpnpm-workspace.yamlinteract with lifecycle behavior. Scoped mode should not paper over these settings.- Do not solve this by globally disabling scripts, adding Python to all downstream shells, or vendoring special cases for
@overeng/pty-effect.
Rollout Plan
- Add a scoped install option to
taskModules.pnpmwhile keeping full-workspace install as default. - Add shell/unit smoke tests for: full-workspace behavior unchanged; scoped install includes internal dependency closure; unrelated workspace lifecycle scripts do not run; lifecycle scripts inside the closure still run; impure flags remain rejected; status hash changes when selected closure changes.
- Add a representative fixture with packages
app -> liband unrelatednative-addon-packagewith a failing install script. - Validate in effect-utils with existing pnpm task tests.
- Switch schickling.dev's nested effect-utils task from the
PNPM_CONFIG_IGNORE_SCRIPTSworkaround to scoped mode. - Run downstream validation in schickling.dev:
CI=1 devenv tasks run check:all. - Remove the workaround from schickling.dev once the new effect-utils input is pinned.
Acceptance Criteria
- Downstream repos can install a selected effect-utils package closure without installing unrelated workspace packages.
- schickling.dev no longer needs
PNPM_CONFIG_IGNORE_SCRIPTSforpnpm:install:effect-utils. @overeng/pty-effect -> @myobie/pty -> node-ptylifecycle scripts are not reached when no selected package depends on that closure.- Lifecycle scripts still execute for packages that are inside the selected closure.
- The shared pnpm task continues to reject
--ignore-scriptsand other impure install flags. - Existing full-workspace
pnpm:installbehavior remains compatible. - Scoped install behavior has smoke coverage in
nix/devenv-modules/tasks/shared/tests/. - CI/devenv diagnostics identify workspace root, selected packages, computed closure, filters, and pnpm log location on failure.
Posted on behalf of @schickling
| field | value |
|---|---|
agent_name |
🍒 co3-cherry |
agent_session_id |
c0b70303-e0c0-447f-b9bb-f47fe1fca13e |
agent_tool |
Codex CLI |
agent_tool_version |
0.131.0 |
agent_runtime |
Codex CLI 0.131.0 |
agent_model |
unknown |
worktree |
schickling.dev/schickling-assistant/own-invoice-matching |
machine |
dev3 |
tooling_profile |
dotfiles@4db6783 |
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in nix/devenv-modules/tasks/shared/pnpm.nix at taskModules.pnpm and inspect the existing shared install, hashing, cleanup, and purity behavior. Then read the smoke tests under nix/devenv-modules/tasks/shared/tests/ and run the existing pnpm task tests. Done means scoped installs preserve full-workspace defaults, select dependency closures without unrelated lifecycle scripts, retain purity and hashing guarantees, and provide the requested diagnostics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- build-system, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100