rust-lang / rust-lang/rust-bindgen

Support `repr(C)` for enums

Open
#3,263 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
5.3k
Forks
829
Avg merge
1d 1h
Merged PRs (30d)
15

Description

Problem

Bindgen supports different enum representations which are either marked repr(transparent) or repr($ty) where $ty is the representation used for the enum (an integer type or an alias for one). However, for cross-language CFI to work correctly, it is sometimes desirable to use repr(C) instead as it causes the compiler to emit the right type ID for cross-language CFI to work.

Example:
enum handler_return {
    INT_NO_RESCHEDULE = 0,
    INT_RESCHEDULE,
};

gives us the following bindings (using --rustified-enum handler_return):

// in context of cross-language CFI, the return type is `core::ffi::c_uint` but
// the expected type is `enum handler_return`) thus an indirect function
// call using this type will abort
#[repr(u32)]
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub enum handler_return {
    INT_NO_RESCHEDULE = 0,
    INT_RESCHEDULE = 1,
}

The following two variations work:

// this works but isn't supported by bindgen as far as I can tell
// (e.g. can't pass `repr(c)` using `--with-attribute-custom-enum ` flag)
#[repr(C)]
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub enum handler_return {
    INT_NO_RESCHEDULE = 0,
    INT_RESCHEDULE = 1,
}

// this also works but requires the user to know about the 
// C++ Itanium ABI/cross-lang CFI internals - not ideal...
#[repr(u32)]
#[cfi_encoding = "14handler_return"]
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub enum handler_return {
    INT_NO_RESCHEDULE = 0,
    INT_RESCHEDULE = 1,
}

Using --newtype-enum handler_return fails the CFI check too:

// fails cfi check
#[repr(transparent)]
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct handler_return(pub core::ffi::c_uint);

and repr(C) or cfi_encoding = "14handler_return" attributes make the CFI check pass again.

Potential Solutions
  1. Add new flags to request repr(C) for enums, e.g. --rustified-enum-repr, --newtype-enum-repr, etc.
  2. Allow --with-attribute-custom-enum REGEX=#[repr(C)]
  3. No action; require the user to use #[cfi_encoding] or change the LLVM cross-language CFI implementation.

The first solution complicates the bindgen CLI but the functionality is easily discoverable. The second solution makes the inverse tradeoff: more can be done with the existing flags but few people will discover that this is the case.

I have prototyped the second approach and can put up a draft PR unless there's a feeling it is not the right way to go. Also very interested in hearing about potentially better solutions I just haven't thought of.

cc @rcvalle @maurer

(Edited to fix mistake pointed out by @emilio in this comment.)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file or test is named. Start by comparing the three proposed approaches with the existing --with-attribute-custom-enum, --rustified-enum, and --newtype-enum options. Done means a supported approach for producing #[repr(C)] on the affected enum representations is decided and its behavior is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, rust
Domain
devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.