rust-lang / rust-lang/rust-bindgen

transparent_union parameters lose the first member signext ABI requirement

Open
#3,415 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
5.3k
Forks
829
Avg merge
1d 1h
Merged PRs (30d)
15

Description

Input C/C++ header
#include <stdint.h>

typedef union __attribute__((transparent_union)) U {
    signed char first;
    signed char second;
} U;

int32_t tu_capture(U value);
Bindgen invocation
$ bindgen input.h \
    --allowlist-type '^U$' \
    --allowlist-function '^tu_.*' \
    --no-layout-tests \
    --rust-target 1.75 \
    --rust-edition 2021 \
    -- -std=gnu11
Actual output
#[repr(C)]
#[derive(Copy, Clone)]
pub union U {
    pub first: ::std::os::raw::c_schar,
    pub second: ::std::os::raw::c_schar,
}

extern "C" {
    pub fn tu_capture(value: U) -> i32;
}

Clang applies the calling convention of the first union member:

define dso_local i32 @tu_capture(%union.U noundef signext %0)

The generated Rust declaration is lowered without signext:

declare noundef i32 @tu_capture(i8)

I used this small x86_64 ABI observer as the implementation:

#include "input.h"

__attribute__((naked, noinline))
int32_t tu_capture(U value __attribute__((unused))) {
    __asm__("movl %edi, %eax\n\t"
            "retq");
}

Calling it from C with .first = -1 produces:

expected=-1
actual=-1

Calling the same object through the generated binding produces:

expected=-1
actual=255

The results are the same at -O0 and -O2.

What is wrong

A transparent_union parameter uses the calling convention of its first
member. Here that member is signed char, so the caller must sign-extend -1
to 0xffffffff. Bindgen emits an ordinary #[repr(C)] union parameter
instead. Rust lowers it as an i8 argument without signext, so the caller
places 0x000000ff in EDI.

The union itself has the correct size and alignment. The mismatch is in the
function parameter ABI.

Expected output

For this declaration, lowering the parameter to the first member type gives an
ABI-correct Rust declaration:

extern "C" {
    pub fn tu_capture(value: ::std::os::raw::c_schar) -> i32;
}

Rust lowers that parameter with signext. If bindgen cannot represent
transparent_union parameters, it should diagnose or omit the affected
function instead of emitting an ordinary union parameter with a different ABI.

Impact

The generated binding compiles and links, but a signed char value of -1
reaches the callee as 255. This reproduces with current main and bindgen
0.72.1.

Environment
bindgen: 0.72.0, current main 25b23474496e78f6a1fbf1c02cb66f11e4d176d0
bindgen release: 0.72.1
clang/libclang: 15.0.7
rustc: 1.75.0
target: x86_64-unknown-linux-gnu
OS: Ubuntu 22.04.5 LTS, x86_64

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with input.h and the shown bindgen invocation, then trace how transparent_union parameters become the generated extern declaration. Done means the first member's ABI requirement is preserved in the generated binding, or the affected function is diagnosed or omitted when that ABI cannot be represented.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, rust
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.