More granular locking in cargo_rustc

View all comments

Summary

Implementation:

• #16307
• #16155

Known issues

• Only non-artifact-producing commands (like check) can run in parallel

Unresolved Issues

• Stabilization of #15010
• Do we need rlimit checks and dynamic rlimits? https://github.com/rust-lang/cargo/pull/16155#discussion_r2632376448
• If there is shared build unit (same ID) with a dirty fingerprint when two checks run, one will block until the other has finished (all units, not just the shared unit) and will rebuild that unit
• Blocking prevents other units from being built on that thread
• Proper handling of blocking message (https://github.com/rust-lang/cargo/pull/16155#discussion_r2589737788)
  • We get the blocking message when taking the initial shared lock, but we get no message when taking the exclusive lock right before compiling.

Future Extensions

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Original issue:

Right now whenever a build happens we lock the entirety of the target directory for the whole build, but it may be possible for us to have a more granular locking strategy which allows multiple instances of Cargo to proceed in parallel instead of serializing them.