rust-lang / rust-lang/cargo

cargo install a library as .dll / .so file

Open
#8,317 9 comments 10 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

C-feature-request Command-install S-needs-design
Dominant language
Rust
Stars
15.5k
Forks
3k
Avg merge
23h 30m
Merged PRs (30d)
51

Description

Describe the problem you are trying to solve

I have a rather large library (azul-dll), which can be compiled to a cdylib. The library itself has 200+ dependencies (already tried to optimize it) and rebuilding code that uses that library takes ~30s, which is slow.

My plan to speed up compile time was to do something like this:

# 1. let the user run an installation command
# optionally, the user can also install the library via apt-get and link it dynamically (not the default)
# compiles and installs the library as a dll into the ./cargo directory
# standardized (?) path for outputting the compiled dll / .so file
cargo install azul-dll -- --release --target=cdylib --output-path="./cargo/installed-libs/azul.so"

Then, in the main azul library, include and load the pre-compiled DLL:

// 2. include the .so file as bytes, so that there are no deployment problems (still one binary file)
// if the dll is not installed, this line will simply fail to compile
#[cfg(feature = "include-dll-in-binary")]
const LIBRARY_BYTES: &[u8] = include_bytes!("./cargo/installed-libs/azul.so");

mod dll {
    pub struct AzulDll {
        lib: Box<Library>,
        az_css_empty: Symbol<extern fn() -> AzCssPtr>,
    }

    fn load_library(bytes: &'static [u8]) -> AzulDll {
        fs::write("/temp/azul.so", bytes); // workaround, since dlopen only accepts file paths
        let lib = Library::new("/temp/azul.so").unwrap();
        AzulDll {
            lib,
            az_css_empty = unsafe { lib.get::<extern fn() -> AzCssPtr>(b"az_css_empty").unwrap() }
        }
    }
}

lazy_static! {
    // initialize the library at startup
    let LIB: AzulDll = load_library(LIBRARY_BYTES);
}

fn main() {
    let css = LIB.az_css_empty();
    println!("{:?}", css);
}

This solution would especially speed up the link time, which is often greater than the compile time itself.

For a release build, the user could specify via feature-flags whether the DLL is loaded from the system and deployed alongside the binary or in a separate package (for example, to work with things like apt on Linux) or if the DLL should be included in the binary for deployment on systems that have little to no library dependency management (Windows), with the disadvantage of slight binary bloat.

Describe the solution you'd like

Currently you can only install binaries from crates.io, it would be great if you could also install DLL / .so files directly from crates.io if they have a crate-type=cdylib in their Cargo.toml.

I could maybe hack around this with giving azul-dll a main.rs file, which the bootstraps itself, and builds and installs azul-dll in the correct directory, but it would be better to have a standardized solution for installing DLLs, especially for security purposes.

Lastly, if something like that would be considered, I've noticed that there is a strip command for cargo now - could the command / rustc be modified to only strip debug information, not function names (which are important for dynamically loading a library)? Thanks in advance for any help.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing cargo install path and the Cargo.toml crate-type=cdylib metadata described in the issue. Compare how DLL and .so output paths, platform deployment modes, and the requested stripping behavior would fit. Done means a settled, secure design for installing shared-library artifacts from crates.io; no implementation files or tests are named.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.