bazel-contrib / bazel-contrib/toolchains_llvm

Please use multithreaded xz

Open
#571 3 comments 5 reactions 0 assignees View on GitHub
Dominant language
Starlark
Stars
371
Forks
283
Avg merge
1d 55m
Merged PRs (30d)
25

Description

My team are constantly complaining that llvm takes over a minute to extract, even if the download is cached. This is due to bazel using single-threaded java to extract .xz archives. It's possible to use an external xz toolchain which uses multithreading, see https://github.com/google-ml-infra/rules_ml_toolchain/commit/50a9847d5fdbca9d8c0351f5a879bfabd6598ad7 for examples

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the referenced rules_ml_toolchain commit, then locate how this repository configures Bazel extraction of .xz archives. The work is complete when .xz extraction uses an external multithreaded xz toolchain instead of single-threaded Java and the relevant behavior is verified.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.