bazelbuild / bazelbuild/rules_rust
Splicing a workspace is slow
- Dominant language
- Starlark
- Stars
- 843
- Forks
- 651
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 15
Description
Sample repo: https://github.com/typesanitizer/slow-repin-lib
`CARGO_BAZEL_REPIN=1 bazel sync --only=crate_index` takes over 2-3 minutes to complete on my M1 Mac, for 169 dependencies. In contrast, a clean `cargo build`, including fetching dependencies, takes about 1m15s.
This example repo is somewhat simplified from our monorepo https://github.com/sourcegraph/sourcegraph, where it takes 150%-250% more time, with ~40% more deps in total.
I understand that some of the overhead may be due to sandboxing on macOS, but I suspect that's probably not the full explanation (it looks like `--sandbox_strategy=local` doesn't work for `bazel sync`, so I couldn't tell how to test without sandboxing).
I don't know if there is a standard way of printing the executed commands (`--subcommands` doesn't work with `bazel sync`), so I hackily added a `print` invocation before the `cargo-bazel` invocation in a local checkout of `rules_rust` and got something like:
```CARGO=/private/var/tmp/_bazel_varun/9733e4cbb184d6ba5c16ee75a5b939da/external/rust_darwin_aarch64__aarch64-apple-darwin__stable_tools/bin/cargo RUSTC=/private/var/tmp/_bazel_varun/9733e4cbb184d6ba5c16ee75a5b939da/external/rust_darwin_aarch64__aarch64-apple-darwin__stable_tools/bin/rustc RUST_BACKTRACE=full CARGO_HOME=/private/var/tmp/_bazel_varun/9733e4cbb184d6ba5c16ee75a5b939da/external/crate_index/.cargo_home /private/var/tmp/_bazel_varun/9733e4cbb184d6ba5c16ee75a5b939da/external/crate_index/cargo-bazel splice --output-dir /private/var/tmp/_bazel_varun/9733e4cbb184d6ba5c16ee75a5b939da/external/crate_index/splicing-output --splicing-manifest /private/var/tmp/_bazel_varun/9733e4cbb184d6ba5c16ee75a5b939da/external/crate_index/splicing_manifest.json --config /private/var/tmp/_bazel_varun/9733e4cbb184d6ba5c16ee75a5b939da/external/crate_index/cargo-bazel.json --cargo /private/var/tmp/_bazel_varun/9733e4cbb184d6ba5c16ee75a5b939da/external/rust_darwin_aarch64__aarch64-apple-darwin__stable_tools/bin/cargo --rustc /private/var/tmp/_bazel_varun/9733e4cbb184d6ba5c16ee75a5b939da/external/rust_darwin_aarch64__aarch64-apple-darwin__stable_tools/bin/rustc --cargo-lockfile /Users/varun/Code/issues/slow-repin/slow-repin-lib/Cargo.lock```
On re-running that command with cargo flamegraph (I'm not sure if I need to be aware of extra caches?), I got this flamegraph: (GitHub doesn't provide interactivity, but you should be able to see more information if you download it from https://user-images.githubusercontent.com/7187503/224541869-3dc2f87e-bbd3-41c7-bcc2-c6c112d5a90a.svg and open it in the browser)

Is there a way to speed up the splicing step on re-pinning?
Contributor guide
Assessment
This issue has not been assessed yet.