Feature: Improve parallel runs
Open
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 816
- Forks
- 88
- PR merge metrics
- No merged PRs in 30d
Description
Hey, when lage is run on multiple machines in the same time. All packages get built on every machine. Because the run is in sync, no package is skipped.
In best scenario each package would only be built once.
- One way to solve this is to use the cache as the messaging system. Where once a machine starts building a package it would save the time it started with the hash. Of course it would need to be a bit more complex. With polling and determining when it is stall.
- In the meantime, a very simple solution would be to randomize the task order (while adhering to topological order). This by itself should improve our build time by a lot.
Thanks 🙏
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the task scheduler and cache implementation, then trace how parallel tasks are ordered and how cache state is recorded. Compare the proposed cache-based coordination with randomized topological ordering; done means concurrent machines avoid redundant package builds without breaking dependency order or stall recovery.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- build-system, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100