rust-lang / rust-lang/libz-sys
Very slow compilation under Windows with libz-ng-sys
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 138
- Forks
- 88
- PR merge metrics
- No merged PRs in 30d
Description
I wasn't quite sure if I should make the issue here or on the flate2 repo, but here we go:
I noticed for a long time that the longest crate to compile on one of my projects is libz-ng-sys that I use through flate2.
I do a lot of testing with different configurations for fun, so I end up cargo cleaning and then rebuilding a lot, so it's particularly bad.
I use the libz backend because the pure rust one is significantly slower, and the problem is even worse on debug.
With a debug full build on said project, on my machine:
windows, libz backend: ~33s
windows, rust backend: ~10s
wsl linux, libz backend: ~14s
Now, I did a bit of investigation on a separate project that only includes libz-ng-sys, by running both commands manually on an empty folder that the build script usually executes, and:
Creating the initial project
This takes ~19s on my system. A big part of it seems to be checking for unixy headers, and the other big part is from checking simd intrinsics. Most of the headers and features are never found for me. This is a lot of time just for creating a project.
Compiling the sources
This takes ~5s on my system. It is not outrageous compared to the first step, but the compilation is single threaded even if the script tries to pass "--parallel" "20". Unfortunately, that's not enough to enable parallel compilation on MSBuild, and it actually actively hurts it. See this SO thread.
Actually passing the multi processing flags correctly brings this down to ~2s, which is better.
Solutions?
The second step can be fixed fairly easily I think, but the first step is a problem so much bigger than the second.
Something in the Windows ecosystem really doesn't play well with CMake, given that the build under Linux is far more reasonable.
Maybe the CMake can be optimized? Maybe ditching CMake completely and just calling cl directly with all the right options? Maybe clang-cl does a better job and we should suggest it to people? Or if the problem is MSBuild, then also have an option for Ninja or something else?
In conclusion, I don't know.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the Windows build timings for libz-ng-sys, including project creation and source compilation through the build script. Compare the CMake/MSBuild path with the mentioned cl, clang-cl, and Ninja options, and inspect how the script passes parallel flags. Done means a concrete, measured reduction in Windows build time without regressing other platforms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, rust
- Domain
- build-system, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100