rust-lang / rust-lang/rust

Using any `repr` other than `u8` on `bool`-like `enum`s loses range information

Open
#122,124 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-codegen C-bug C-optimization I-slow S-has-mcve T-compiler
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

I tried this code:

#[repr(u32)]
#[derive(Clone, Copy)]
pub enum E {
    A = 0,
    B = 1,
}

#[no_mangle]
fn f(e: E) -> i32 {
    match e {
        E::A => 1,
        E::B => -1,
    }
}

#[no_mangle]
fn g(e: E) -> E {
    match e {
        E::A => E::B,
        E::B => E::A,
    }
}

I expected to see this assembly:

f:
        mov     eax, edi
        neg     eax
        or      eax, 1
        ret

g:
        mov     eax, edi
        xor     al, 1
        ret

Instead, I got this assembly:

f:
        xor     eax, eax
        neg     edi
        sbb     eax, eax
        or      eax, 1
        ret

g:
        xor     eax, eax
        test    edi, edi
        sete    al
        ret

godbolt good
godbolt bad

The problem seems to be that bool-like enums which are not repr(u8) are passed to functions as i*, while repr(u8) are passed as i1 zeroext. The lacking range information then cascades into missed optimizations whenever you try to do something with the parameter.

Notes:

  • g gets optimized when adding an unreachable_unchecked() OR an unreachable!(), but f is not so lucky (godbolt)
  • other archs are also affected
    • aarch64 suffers slightly (godbolt)
    • wasm arguably benefits, at least in code compactness (godbolt)
Meta

rustc --version --verbose:

rustc 1.78.0-nightly (d18480b84 2024-03-04)
binary: rustc
commit-hash: d18480b84fdbf1efc34f62070951334aa833d761
commit-date: 2024-03-04
host: x86_64-unknown-linux-gnu
release: 1.78.0-nightly
LLVM version: 18.1.0
Compiler returned: 0

@rustbot modify labels: +C-optimization +A-codegen +I-slow

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Rust examples for f and g and comparing their generated assembly on the linked Godbolt cases. Trace how rustc and LLVM represent bool-like enums with non-u8 reprs, then verify that the relevant range information is preserved and the expected optimizations appear across the affected architectures.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.