bazelbuild / bazelbuild/rules_rust

Rustc improvements for C++ linking

Open
#637 2 comments 3 reactions 1 assignee Claimed by @hlopko View on GitHub
bug needs-triage
Dominant language
Starlark
Stars
843
Forks
651
Avg merge
2d 18h
Merged PRs (30d)
15

Description

# Overview

`rustc` lacks certain features with regards to linking that prohibit us from
using `rustc` as the linker driver in `rules_rust` for a tight integration with a large existing C++ codebase.

# Problem description

In general `rustc` supports 2 ways of configuring linking:

+ [Source code attribute](https://doc.rust-lang.org/reference/items/external-blocks.html#the-link-attribute)
`#[link]`
+ [Command line argument](https://doc.rust-lang.org/rustc/command-line-arguments.html#-l-link-the-generated-crate-to-a-native-library)
`-l:`

In Bazel we use the latter, but feature-wise they are equivalent.
To declare a dependency on a static library `libfoo.a`, we pass `-l:static=foo`
to `rustc`. This will currently link `libfoo.a` as `alwayslink`, in a
`--whole-archive/--no-whole-archive` block. As a result, all symbols from
`libfoo.a` are seen as used by the linker. While there are some
C++ libraries that have to be linked as `alwayslink`, the vast majority of libraries
don't want that - it increases the binary size, the link time, and can even
uncover missing symbol bugs not encountered in regular C++ builds.

Even the fact that `rustc` expects static archives is a problem for certain C++
toolchains. Some toolchains don't typically create static archives, instead they
pass object files directly to the linker in a `--start-lib`/`--end-lib` block.
This saves some storage (archives pretty much duplicate the disk used by object
files), and saves io between developer machine and remote cache. This is not
supported by `rustc`.

Some libraries don't match the `lib.a` format. For example some projects
use `lib.lo` for `alwayslink` libraries. Rustc doesn't support linking to
nonstandard (or verbatim) names using `-l:` flag currently. There is a way to specify flags
that will be passed verbatim to the linker on the `rustc` command line -
`--codegen=link-arg`. However `rustc` will put these flags at the end of the
linker command line, after it passes crate libraries and Rust standard and core
libraries. That can result in backward references. Imagine a `Crate A` depends
on C++ `Native B`, which depends on `Crate C`. Linker would ideally see
`-lcrateA -lnativeB -lcrateC -lrust_stdlib -lc++`. But because `link-arg` flags
are moved at the end, it sees `-lcrateA -lrust_stdlib -lnativeB -crateC -lc++`.
The backwards reference problem is triggered when `Crate C` depends on a symbol
from the Rust standard library.

# Unstable features/accepted RFCs

## static-nobundle

+ Tracking issue
[https://github.com/rust-lang/rust/issues/37403](https://github.com/rust-lang/rust/issues/37403)

This unstable feature allows us to disable `alwayslink` linking behavior by
using the `-l:static-nobundle=foo.a` flag. It doesn't solve the nonstandard
naming and object group problems. In addition, it seems it's made obsolete by
the `native-link-modifiers` RFC.

## native-link-modifiers

+ Tracking issue
[https://github.com/rust-lang/rust/issues/81490](https://github.com/rust-lang/rust/issues/81490)
+ RFC
[https://rust-lang.github.io/rfcs/2951-native-link-modifiers.html](https://rust-lang.github.io/rfcs/2951-native-link-modifiers.html)

This not yet implemented RFC solves all above mentioned problems.

# What to do before RFC 2951 is implemented?

Until RFC 2951 is implemented we don't recommend using `rust_binary` for mixed C++/Rust binaries.

Current workaround for legacy C++ codebases is to use `cc_binary` to drive the final
transitive linking.

0) make sure your `rust_toolchain` declares core and std rlibs so `cc_binary` knows what to link (TODO: hlopko should upload alloc trampolines that durin42 implemented).
1) don't use `rust_binary` for the binary crate, use `rust_library`
2) set `crate_root` in the `rust_library` to point to the `main.rs`
3) add a `cc_binary` that depends on the `rust_library`

+ in `main.rs`, replace `fn main() {...}` with

```rust
#[no_mangle]
extern "C" fn main() {...}
```

4) build

We believe we can implement a Starlark rule/macro that will automate the whole process. We'll attempt that in the near future.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.