no-op remap_path_prefix change invalidates incremental cache
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
I'm building rust code in a CI environment, setting -Cincremental to a shared location. CI allocates a new working directory for each build. To prevent leaking that info to the debug info, we set --remap_path_prefix=${PWD}= to eliminate the unwanted prefix from the debug info.
rustc src/lib.rs \
--crate-name=xxx \
--crate-type=rlib \
--error-format=human \
--out-dir=${OUTDIR} \
--remap-path-prefix=${PWD}= \
--emit=dep-info,link \
-Ccodegen-units=256 \
-Zincremental_info \
-Cincremental=${CACHEDIR}
I expected this will reuse the incremental build cache, but it's not:
[incremental] session directory: 259 files hard-linked
[incremental] session directory: 0 files copied
[incremental] completely ignoring cache because of differing commandline arguments
Further inspection of how the hash being generated, I found:
for our case, working_dir always uses remapped_path "", which is stable over build. but the remap_path_prefix is different everytime. and the logic for populating working_dir is at:
So to get correct remapped working_dir, I must set --remap-path-prefix, but setting it will change remap_path_prefix and causing hash mismatch:
Meta
rustc --version --verbose:
rustc 1.82.0-nightly (f6e511eec 2024-10-15)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: aarch64-apple-darwin
release: 1.82.0-nightly
LLVM version: 19.1.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the option hashing in compiler/rustc_session/src/options.rs, the remapped working-directory setup in compiler/rustc_session/src/config.rs, and cache validation in compiler/rustc_incremental/src/persist/load.rs. Reproduce the shared incremental-cache build using changing --remap-path-prefix values, then trace which command-line inputs affect the hash. Done means a no-op remap change no longer prevents valid cache reuse, with regression coverage for the reported scenario.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100