rust-lang / rust-lang/rust

Rust reproducibility issue - Finding the proper fix

Open
#134,589 12 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-reproducibility C-discussion T-compiler
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

We are building Rust for a custom target in the Yocto framework. Here we run a test called reproducibility to ensure the rust binaries are identical between the builds.
We do that by building and comparing the rust binaries & libs in two different build directories -

  1. reproducibleA &
  2. reproducibleB-extended

With every rust release we've seen a several issues which makes the rust binaries are not reproducible. The issues seen are with several reasons like -

  1. Regressions caused by rust updates (absolute path in proc-macro)
    https://github.com/rust-lang/rust/pull/121959/commits/a9a979839bbdfec48c75d618ab0dce8a953589b8

  2. Incorrect/missing Rust config settings (remap-debuginfo is not set by default)
    https://lists.openembedded.org/g/openembedded-core/message/188940

  3. Compiler options and settings - rustdoc failed with codegen-units option setting
    https://lists.openembedded.org/g/openembedded-core/message/202540

  4. Regressions with Crate updates - CC crate appending a unique hash to the object files
    https://github.com/rust-lang/cc-rs/pull/1277
    https://github.com/rust-lang/cc-rs/pull/1270

There are a few other failure cases we've analyzed and fixed. This issue is happening for multiple reasons and these failure are resulting in lot of debug efforts and project delays.

This can be avoided by having such a reproducibility test on rust itself.

Any comments or inputs on detecting and fixing this issue during rust development itself? (via a test case or any other ways)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the reproducibility failures described for custom targets and Yocto builds, including absolute paths in proc-macros, remap-debuginfo, rustdoc codegen-units, and crate updates. Define the scope of a Rust-side reproducibility test and determine which build outputs and configurations it should compare. Done means the test reliably detects the listed classes of non-reproducible output.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.