Layout optimization fails for enums with variants having values 0 and 255
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
When a C-like enum has two variants, one being set to 0 and the other being set to 255 the layout optimizations fail to apply even though they do apply if the values are different.
I tried this code:
struct Foo {
_bar: Bar,
_baz: [u8; 7],
}
enum Bar {
_A = 0,
_B = 255,
}
struct Error {
_x: bool,
_y: bool,
}
fn main() {
println!("{}", core::mem::size_of::<Result<Foo, Error>>());
}
I expected to see this happen: 8 being printed because of layout optimization, since all values 1..=254 are invalid the optimizer can pick any of them to represent the Err variant and store the error data in the space that would be otherwise occupied by [u8; 7]. This already works if the values are different.
Instead, this happened: The output is 9, implying the optimization was not applied. If you change the value of _A or _B then the optimization is applied (prints 8).
Meta
rustc --version --verbose:
rustc 1.86.0-nightly (ae5de6c75 2025-01-29)
binary: rustc
commit-hash: ae5de6c759cd337ecdb2de4e94f47eaafb5d4606
commit-date: 2025-01-29
host: x86_64-unknown-linux-gnu
release: 1.86.0-nightly
LLVM version: 19.1.7
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by compiling the provided Rust reproducer and checking its reported Result<Foo, Error> size. Trace the enum layout optimization for variants with values 0 and 255; the issue is resolved when this case produces the optimized size of 8 without regressing cases with different values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100