dotCMS / dotCMS/core

Build: adopt pnpm virtualStoreType: global for git worktrees

Open
#37,573 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dotCMS : Build Team : Falcon Type : CI/CD
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Description

pnpm's git-worktrees guide recommends virtualStoreType: global
(spelled enableGlobalVirtualStore: true before 11.23.0): each worktree's node_modules holds
symlinks into one shared content-addressable store instead of materialising its own copy.

Measured in core-web on the pnpm 12 branch (#37563), same commit and lockfile, only the setting
changed:

virtualStoreType: global without
core-web/node_modules 2.2 MB 2.1 GB
pnpm install --frozen-lockfile (warm) 4.3s 15.1s
lockfile unchanged unchanged
nx build dotcms-ui fails green, 38s

~2.1 GB per checkout is the prize, and several of us keep 5-10 worktrees of this repo open at
once. It is a local-development win only — a CI job is a clean machine with a single checkout.

It was left out of #37563 because it breaks the build in a way we cannot fix from our own
configuration. This issue records the investigation so it does not have to be repeated.

What breaks

nx build dotcms-ui --skip-nx-cache fails during bundling. Every error is a package failing to
resolve an import, and every path is inside the global store:

✘ [ERROR] Could not resolve "yjs"
    .../store/v11/links/@/y-protocols/1.0.1/<hash>/node_modules/y-protocols/awareness.js
✘ [ERROR] Could not resolve "rxjs" / "rxjs/operators"
    .../store/v11/links/@tinymce/tinymce-angular/7.0.0/<hash>/.../tinymce-tinymce-angular.mjs
✘ [ERROR] Could not resolve "@angular/forms"
    .../store/v11/links/@materia-ui/ngx-monaco-editor/6.0.0/<hash>/...
✘ [ERROR] Could not resolve "chart.js/auto"

Root cause

The failing packages import modules they do not declare:

  • y-protocols@1.0.1 imports yjs; its manifest declares only lib0 and has no
    peerDependencies at all.
  • @tinymce/tinymce-angular@7.0.0 imports rxjs; its manifest declares tinymce and tslib,
    and lists @angular/common, @angular/core, @angular/forms as peers.

Under the local virtual store these resolve by accident of geometry. From
node_modules/.pnpm/y-protocols@1.0.1/node_modules/y-protocols/, Node's walk-up goes:

node_modules/.pnpm/y-protocols@1.0.1/node_modules/   → lib0 (declared dep)
node_modules/.pnpm/node_modules/                     → hoisted fallback (1202 packages here)
node_modules/                                        → yjs, rxjs (direct deps of core-web)

The last step is the rescue: an undeclared import is satisfied because the walk eventually
reaches the project's own node_modules.

Under the global virtual store the package lives at ~/Library/pnpm/store/v11/links/..., so
walking up leaves the project entirely and never reaches core-web/node_modules. Nothing rescues
the undeclared import.

This is inherent to placing the virtual store outside the project directory, not a setting we
have failed to find.

preserveSymlinks was tried and does not work

The obvious lead — @angular/build:application accepts preserveSymlinks, and it is not set
anywhere in this repo — makes the build fail the other way:

✘ [ERROR] Could not resolve "lib0/iterator"
    node_modules/yjs/dist/yjs.mjs:17:26

With preserveSymlinks: true, resolution walks up the symlinked path to core-web/node_modules,
where transitive dependencies correctly do not exist — that is pnpm's strict isolation
working as designed. So the default breaks undeclared imports and preserveSymlinks breaks
transitive ones. Both directions fail, for opposite reasons. Do not spend time re-testing this.

What an actual fix would take

  1. Patch the offending packages so they declare what they import, via
    pnpm.patchedDependencies or overrides. At least four are implicated
    (y-protocols, @tinymce/tinymce-angular, @materia-ui/ngx-monaco-editor, and whatever
    pulls chart.js/auto), and the only way to find the rest is iterating on failed builds.
  2. Or wait for upstream to provide a fallback reachable from the global store.

Either is its own project with its own verification cycle, which is why it is not part of the
pnpm 12 migration.

If someone picks this up

Beyond getting the build green:

  • dist/apps/dotcms-ui/tinymce and dist/apps/dotcms-ui/assets/monaco-editor asserted
    non-empty by file count, not by exit code. They are recursive glob: **/* copies out of
    a directory that becomes a symlink pointing outside the workspace root. Baseline on a
    working build: 217 files / 10 MB and 1068 files / 84 MB.
  • nx run-many -t test green, including ui and edit-content
  • pnpm nx serve dotcms-ui with no /@fs/ 403s — no server.fs.allow is configured
    anywhere in the repo and Vite applies that guard to realpaths
  • unrs-resolver exercised on CI, not only locally — already documented as segfaulting
    inside @nx/vite graph inference (libs/sdk/vue/vite.config.mts)
  • CI store cache still hits: the workflow caches pnpm store path only and never caches
    node_modules, so confirm the global virtual store does not land outside that path
  • Always verify with --skip-nx-cache. A cached run replays the recorded output of
    pnpm exec stencil build and reports success without executing anything.

Additional Context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing nx build dotcms-ui --skip-nx-cache with the global virtual store and inspect the listed unresolved packages. Review the package manifests for y-protocols, @tinymce/tinymce-angular, @materia-ui/ngx-monaco-editor, and the package pulling chart.js/auto. Done means the build and nx run-many -t test pass, copied assets are non-empty, serve has no /@fs/ 403s, and CI cache behavior is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
angular, typescript
Domain
build-system, ci-cd, frontend, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.