rustc and clang differ on noundef for hidden sret pointer parameters causing MSAN failures
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
Running the below code in msan (-Zsanitizer=memory) fails MSAN. A simple Cargo-based repro case can be found at https://github.com/th0br0/bug-rustc-cc-interop-msan.
Failure:
==3140423==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 ...
companion.cc:
struct Large {
uint64_t a[3];
};
extern "C" Large cpp_return_large(uint64_t x) {
return Large{x, 0, 0};
}
main.rs
#[allow(dead_code)]
pub enum Padded {
Small([u8; 1]),
Large([u32; 1]),
}
#[repr(C)]
pub struct Large(pub [u64; 3]);
unsafe extern "C" {
pub fn cpp_return_large(x: u64) -> Large;
}
#[inline(never)]
pub fn pass_padded(_: Padded) {}
fn main() {
pass_padded(Padded::Small([1]));
let large = unsafe { cpp_return_large(42) };
}
Automated root causing found that in rustc
https://github.com/rust-lang/rust/blob/main/compiler/rustc_target/src/callconv/mod.rs#L419 and https://github.com/rust-lang/rust/blob/main/compiler/rustc_codegen_llvm/src/abi.rs#L108 emit noundef but clang does not https://github.com/llvm/llvm-project/blob/main/clang/lib/CodeGen/CGCall.cpp#L3085 this then leads to this LLM-generated step-by-step:
- Step A: Rust calls a function by value with an 8-byte enum containing padding bytes (e.g.
enum Padded { Small([u8; 1]), Large([u32; 1]) }, passed viaPassMode::Cast(i64)withoutnoundef), which stores non-zero padding shadow(0xffffff00_00000000)into__msan_param_tls[0]. - Step B: Rust calls a Clang-compiled
extern "C"function that returns a> 16-byte struct viasret (Arg#0, %rdi).Because rustc marks thesretargumentnoundef, LLVM'sMemorySanitizerPass::visitCallBaseskips writing0to__msan_param_tls[0]. - Step C: Because Clang omits
noundefon thesretparameter, MemorySanitizerPass in the C/C++ callee loads the stale shadow from__msan_param_tls[0]as the shadow for thesretpointer(%rdi)and traps when storing fields into*sret.
This supposedly can be fixed in either clang by emitting noundef or rust omitting noundef. I don't know which would be preferable.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked Cargo repro and run it under MSAN to confirm the failure. Read rustc_target/src/callconv/mod.rs around line 419 and rustc_codegen_llvm/src/abi.rs around line 108, then compare the noundef handling with Clang's CGCall.cpp around line 3085. Done means establishing the compatible ABI behavior and demonstrating that the repro no longer reports the stale-shadow MSAN failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100