[RRFC] Create dedicated lockfile for a specific workspace

Motivation ("The Why")

I'm working on a large monorepo with a few dozen packages, using npm workspaces. A lot of these packages are frontend applications and backend services, each of which are deployed as docker containers. To do so, the build process builds a docker image for each of them and has to install the required dependencies in the process. To do so, we copy the root-level package-lock.json and the individual package.json of that package to the filesystem of the container and perform a npm installation afterwards.

The interesting part of all of the Dockerfile looks like this (simplified):

COPY package-lock.json .
COPY service/package.json .
RUN npm ci --audit false --fund false

COPY service/ .
RUN npm run build

This works fine and will only install the dependencies required for this exact package 👍🏻

However, I have more than three dozen workspaces in the monorepo and update all dependencies in all workspaces automatically (think renovate, dependabot etc). This means the root package-lock.json is updated multiple times a day - which leads to a problem.

Due to the COPY package-lock.json being one of the first instructions in all Dockerfile, this file changing multiple times a day prevents us from efficiently using dockers build cache: Every time any workspace updates a dependency, all cached docker build layers after this instruction are invalidated and have to be re-executed from scratch for every workspace package.

Both yarn (userland) and pnpm have tools to generate a specific lockfile for a single workspace out of the hoisted root lockfile.

It should be possible to also do this with npm - but I couldn't find any readymade solutions for it.

What do you think?

Example

It would be great if there was a CLI Command that would generate a package-lock.json for a specific workspace out of the hoisted root package-lock.json. In the absence of a better name, let's call it "make-dedicated-lockfile". Running npm make-dedicated-lockfile in workspace "service" or running npm make-dedicated-lockfile -w service in the project root would generate a dedicated package-lock.json in the "service" workspace to reflect the reproducible dependency tree of that specific workspace.

How

Current Behaviour

I need to use package-lock.json and run npm ci to ensure that installations remain identical and reproducible throughout the entire dependency tree across all the systems and pipelines, one of them being building a docker image for each workspace.

Right now, this is done with a npm script in each workspace:

docker build .. -f Dockerfile

Desired Behaviour

Before building a docker image for each workspace, generate a dedicated lockfile specific to that workspace. Build the docker image, then delete the specific lockfile.

This could be done with a npm script like this:

npm make-dedicated-lockfile && docker build . -f Dockerfile && rm package-lock.json

The Dockerfile would then copy over the dedicated lockfile instead of the root level one (please take note of the context switch in the docker npm script. Context was previously set to .. (repository/workspace root), and can now be set to . (workspace folder).

COPY package-lock.json .
COPY package.json .
RUN npm ci --audit false --fund false

COPY / .
RUN npm run build

Additional thoughts

It is possible that you might not want multiple package-lock.json files in a workspaces project, ever - because it might confuse people. If that's the case, one could think of a few ways to solve this - for example generating a dedicated-lock.json file instead and adding a command line flag to npm ci that would need to specifically point to this dedicated lockfile:

npm make-dedicated-lockfile && docker build . -f Dockerfile && rm dedicated-lock.json

COPY dedicated-lock.json .
COPY package.json .
RUN npm ci --audit false --fund false --use-workspace-lockfile

COPY / .
RUN npm run build

Start by reviewing the workspace package-lock.json and package.json flow described in the proposal, along with the npm commands and Dockerfile examples. Done would mean a selected workspace can produce a dedicated reproducible lockfile from the hoisted root lockfile, including the proposed root -w and workspace invocations, for use with npm ci.