suggest compile binaries with dynamic libs
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
when I compile benchmarks binaries with `cargo build --release` and use `mold` as the linker, it occurs following errors
```
(signal: 9, SIGKILL: kill)
warning: build failed, waiting for other jobs to finish...
error: could not compile `datafusion-benchmarks` (bin "dfbench")
...
(signal: 9, SIGKILL: kill)
Building [=======================> ] 443/446: imdb(bin), tpch(bin), external_aggr(bin)
```
and the errors disappeared when I run `CARGO_BUILD_JOBS=1 cargo build --release`
but each binary took about 5 to 10 min to compile, it's too slow.
```
Finished `release` profile [optimized] target(s) in 25m 01s
```
I found the binaries share many rlib files, such as libdatafusion_benchmarks-bb94f26d75d1c935.rlib, could you compile them into .so files, and let the smaller binary load the .so files at runtime? it can save many time and CPU.
### Describe the solution you'd like
compile them into .so files, and let the smaller binary load the .so files at runtime
### Describe alternatives you've considered
supply the binaries when publish the new release
### Additional context
_No response_
Contributor guide
Research direction
Start by reproducing the `datafusion-benchmarks` release build with `cargo build --release` and `mold`, comparing parallel compilation with `CARGO_BUILD_JOBS=1`. Investigate how the benchmark binaries share rlib files and define a build approach that produces reusable dynamic libraries without breaking runtime loading; done means the release build avoids repeated compilation while the binaries still run correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100