[Bug?]: workspaces run concurrency
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 8.1k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Self-service
- I'd be willing to implement a fix
Describe the bug
See the following command, where as I understand should run the build script in all the packages listed in dependencies or devDependencies in topological order, but it fails if some of the packages mentioned has not a build script while it list other dependencies that should be build beforehand too.
yarn workspaces foreach --recursive --topological-dev --from packageA run build
packageA
dependencies:
packageB
dependencies:
packageC
scripts:
build: "tsc ..."
scripts:
build: "tsc ..."
As packageB has not a build script it DOES NOT WAIT FOR the build of packageC going into a race condition.
To workaround, we can add to packageB a dumb script "build": "true". In my opinion, it is a bug, it does not match the expectation (see docs):
Yarn will only run the command after all workspaces that it depends on through the dependencies field have successfully finished executing.
To reproduce
const {promises: {readFile, writeFile}} = require("fs");
const path = require("path");
const file = path.resolve("output.log");
await writeFile(file, `
`);
await packageJson({
name: "root",
private: true,
workspaces: ["packages/*"],
});
await packageJson(
{
name: "packageA",
version: "1.0.0",
dependencies: {
"packageB": "*",
},
scripts: {
build: "echo '** A **' >> " + file
},
},
{ cwd: "packages/packageA" }
);
await packageJson(
{
name: "packageB",
version: "1.0.0",
dependencies: {
"packageC": "*",
},
// if the package has not the build script then it DOES NOT block
// scripts: {
// build: "echo '** B **' >> " + file
// },
// a workaround is to use a dummy script, in this case it DOES block
// scripts: {
// build: "true"
// },
},
{ cwd: "packages/packageB" }
);
await packageJson(
{
name: "packageC",
version: "1.0.0",
scripts: {
build: "echo '** C **' >> " + file
},
},
{ cwd: "packages/packageC" }
);
await writeFile(".yarnrc.yml", `
nodeLinker: node-modules
`);
await yarn("install");
await yarn("workspaces", "foreach", "--recursive", "--topological-dev", "--from", "packageA", "run", "build");
const output = await readFile(file, "utf8");
// when packageB HAS NOT a build script, the topology IS NOT correct, building: A > C
expect(output).toContain(`
** C **
** A **
`);
// when packageB has a build script, the topology is correct, building: A < B < C
// expect(output).toContain(`
// ** C **
// ** B **
// ** A **
// `);
Environment
System:
OS: macOS 12.1
CPU: (10) arm64 Apple M1 Pro
Binaries:
Node: 16.13.1 - /private/var/folders/2t/vh754bvs45303cbdj6b063jc0000gp/T/xfs-e051e464/node
Yarn: 3.1.1 - /private/var/folders/2t/vh754bvs45303cbdj6b063jc0000gp/T/xfs-e051e464/yarn
npm: 8.1.2 - /opt/homebrew/Cellar/node@16/16.13.1/bin/npm
Additional context
If the build of packageC is very slow (imagine several minutes), we will be running code in packageA with stale version of packageC or without if it was cleaned early.
broken chain:
A -- import --> B -- import --> C
// packageA
import foo from 'packageB';
// packageB
export foo from 'packageC';
// packageC
export default 123;
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the supplied JavaScript repro, the generated package.json files, .yarnrc.yml, and the yarn workspaces foreach --recursive --topological-dev entry point. Verify the ordering with packageA, packageB, and packageC when packageB lacks a build script; done means packageA waits for packageC as required by the documented dependency topology.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs, typescript
- Domain
- build-system, cli, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100