yarnpkg / yarnpkg/berry

[Bug?]: `yarn rebuild <module>` still rebuilds all other modules that haven't previously been built

Open
#3,166 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug reproducible
Dominant language
TypeScript
Stars
8.1k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Self-service
  • I'd be willing to implement a fix
Describe the bug

If I have two dependencies with post-install scripts, e.g. esbuild and xsd-schema-validator, running yarn rebuild esbuild will cause the post-install script for xsd-schema-validator to also be run. Per the docs:

By default all packages will be rebuilt, but you can filter the list by specifying the names of the packages you want to clear from memory.
Rebuild fsevents only :

yarn rebuild fsevents

Note that this only happens if xsd-schema-validator has not been built yet - if it's last build failed or no attempt has been made to build it previously.

The only reason I'm trying to build a select list of modules is that I'm working on a monorepo where builds are disabled for regular yarn install (on a developer's machine) but are needed in other environments e.g. in Dockerfiles. Additionally, the Dockerfiles only build certain modules that are needed in the eventual container. There's currently no feature that allows building certain modules in certain environments - a combination of enableScripts: false and dependenciesMeta only allows configuring a list of modules to build in all environments. Also, I'd prefer to not use dependenciesMeta for a deny-list as that is not very maintainable.

To reproduce

Playground

process.env.YARN_ENABLE_SCRIPTS = 'false';

const installPromise = packageJsonAndInstall({
dependencies: {
[`esbuild`]: `0.12.11`,
[`xsd-schema-validator`]: `0.7.0`,
},
});

await expect(installPromise)
.resolves.toContain(`xsd-schema-validator@npm:0.7.0 lists build scripts, but all build scripts have been disabled`);

process.env.YARN_ENABLE_SCRIPTS = 'true';

const rebuildPromise = yarn(`rebuild`, `esbuild`)

await expect(rebuildPromise)
.resolves.not.toContain(`xsd-schema-validator`);

Output:

Error: expect(received).resolves.not.toContain()

Received promise rejected instead of resolved
Rejected to value: [Error: Command failed: /home/sandbox/.nvm/versions/node/v14.16.1/bin/node /sandbox/berry/scripts/actions/../run-yarn.js rebuild esbuild·
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0007: │ esbuild@npm:0.12.11 must be built because it never has been before or the last one failed
➤ YN0007: │ xsd-schema-validator@npm:0.7.0 must be built because it never has been before or the last one failed
➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDOUT [xsd-schema-validator] Compiling helper...
➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR Error: Java SDK required at JAVA_HOME or in path to compile validation helper
➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR     at /tmp/tmp-61LAIR84tWXcQs/.yarn/unplugged/xsd-schema-validator-npm-0.7.0-172763c240/node_modules/xsd-schema-validator/lib/validator.js:61:9
➤ YN0009: │ xsd-schema-validator@npm:0.7.0 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-aa2d3681/build.log)
➤ YN0000: └ Completed in 0s 290ms
➤ YN0000: Failed with errors in 0s 352ms
]
at expect (/sandbox/node_modules/expect/build/index.js:138:15)
at module.exports (evalmachine.<anonymous>:18:7)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async /sandbox/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
at async executeInTempDirectory (/sandbox/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
at async executeRepro (/sandbox/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
at async module.exports../pages/api/sherlock.js.__webpack_exports__.default (/sandbox/.next/server/static/development/pages/api/sherlock.js:140:29)
at async Object.apiResolver (/sandbox/node_modules/next/dist/next-server/server/api-utils.js:46:9)
at async DevServer.handleApiRequest (/sandbox/node_modules/next/dist/next-server/server/next-server.js:449:9)
at async Object.fn (/sandbox/node_modules/next/dist/next-server/server/next-server.js:371:37)
at async Router.execute (/sandbox/node_modules/next/dist/next-server/server/router.js:134:32)
at async DevServer.run (/sandbox/node_modules/next/dist/next-server/server/next-server.js:491:29)
at async DevServer.handleRequest (/sandbox/node_modules/next/dist/next-server/server/next-server.js:144:20)

Environment

System:
OS: Linux 5.4 Ubuntu 20.04.2 LTS (Focal Fossa)
CPU: (4) x64 Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz
Binaries:
Node: 14.15.1 - /tmp/xfs-2343c956/node
Yarn: 3.0.0-rc.12.git.20210723.hash-b2a97475 - /tmp/xfs-2343c956/yarn
npm: 6.14.8 - ~/.nvm/versions/node/v14.15.1/bin/npm

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Playground reproduction and inspect the yarn rebuild esbuild path. Confirm that the rebuild command currently schedules xsd-schema-validator when it has never been built; done means the selected package is rebuilt without triggering unrelated pending build scripts, with the reproduction passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
build-system, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.