Early-building some parts of graph
- Dominant language
- Rust
- Stars
- 4.4k
- Forks
- 394
- PR merge metrics
- No merged PRs in 30d
Description
Thanks for the incredible work. I have a question about action build order that I'm looking for some insight on:
Say I have a build graph that looks like:
actions 0-100 --------------------> \\
``````````````````````````````````` -----> cli `//:target` I asked for
action 101 --> actions 102-200 ---> /
Where all of these actions are CPU-bound but where `action 101` can't saturate the available CPU / is not perfectly parallel. In my case this is a bunch of C objects that can be compiled right away, plus a bunch of C objects that depend on the output of a code generator first, and then combining those into a binary I ask for on the command line with `buck2 build`.
Today, I notice buck2 does not look like it builds `action 101` with some kind of priority given that it's blocking off a large part of the action graph. If `action 101` takes a while, we sit there waiting for it somewhere in the "middle" of the build, whereas we could have frontloaded it by building it earlier in the build. I can do this: (`//:long_step` builds action 101):
`buck2 build //:long_step //:target`
And buck2 does start running `action 101` earlier and the build finishes faster overall because less time is spent with <100% resource utilization waiting for a blocking action to complete and unblock the rest of the build.
Does buck2 know how to execute the graph in this kind of order where we prioritize steps that block larger numbers of other steps? This makes a material impact to build times; 24 -> 19 seconds in an actual project. I can make a reproducer if you'd like.
Contributor guide
Assessment
This issue has not been assessed yet.