rust-lang / rust-lang/rustc_codegen_gcc

Bug and fix: `core::fmt` hangs with formatting flags (`{:#x}`, `{:#o}`, `{:#b}`, etc.)

Open
#871 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
1.2k
Forks
105
Avg merge
8h 20m
Merged PRs (30d)
14

Description

Symptoms

When using formatting flags with an alternate flag or alignment/padding, the program hangs, produces no output, and raises no exception:

println!("{:#x}", 42u32);    // hangs
println!("{:#010x}", 42u32); // hangs
println!("{:>10}", 42u32);   // hangs
println!("{:x}", 42u32);     // works normally
println!("{}", 42u32);       // works normally
Root Cause

bswap.rs in compiler_builtins uses #[maybe_use_optimized_c_shim] + x.swap_bytes():

rust
// Original code (problematic)
#[maybe_use_optimized_c_shim]
pub extern "C" fn __bswapsi2(x: u32) -> u32 {
    x.swap_bytes()   // ← rustc_codegen_gcc compiles this into a call to __bswapsi2 itself
}
rustc_codegen_gcc compiles u32::swap_bytes() into a function call to __bswapsi2, leading to:

text
__bswapsi2 → ___bswapsi2 → __bswapsi2 → ___bswapsi2 → ... (infinite recursion)
After stack overflow, the PC jumps to an invalid address (e.g., 0xfb079370), causing the program to hang.

Inside core::fmt, handling the alternate flag (#) calls pad_integral, which in turn triggers __bswapsi2, exposing this bug.

Debugging Process

QEMU -d exec revealed the program stuck at 0x0dfc (OpenRISC exception vector area)

Vector 0xC00 = syscall exception, triggered by l.sys 0x8060

GDB breakpoint at 0xC00, ppc = 0x26914, located in the __bswapsi2 call chain

Disassembly showed mutual recursion: __bswapsi2 → ___bswapsi2 → __bswapsi2

Files to Modify

The bswap.rs files in the following three paths have identical content (same md5) and all need modification:

text
~/or1k-build/rustc_codegen_gcc/build/build_sysroot/sysroot/lib/rustlib/src/rust/library/compiler-builtins/compiler-builtins/src/int/bswap.rs
~/or1k-build/rustc_codegen_gcc/build/build_sysroot/sysroot_src/library/compiler-builtins/compiler-builtins/src/int/bswap.rs
~/or1k-build/rust-src/library/compiler-builtins/compiler-builtins/src/int/bswap.rs
The actual path used by run.sh (with -Z build-std=core) is the stage1 sysroot path, which can be confirmed with:

bash
find ~/or1k-build/rust-src/build -name "bswap.rs" -path "*/compiler-builtins/*" 2>/dev/null
Fix

Replace x.swap_bytes() with pure shift operations to eliminate recursion:

bash
# Backup (all three paths have identical content, backup together)
find /home/openrisc/or1k-build -name "bswap.rs" -path "*/compiler-builtins/*" | \
    while read f; do cp "$f" "${f}.bak"; echo "Backed up: $f"; done

# Modify (apply to all three paths)
find /home/openrisc/or1k-build -name "bswap.rs" -path "*/compiler-builtins/*" \
    ! -name "*.bak" | while read f; do
cat > "$f" << 'BSWAP_EOF'
intrinsics! {
    pub extern "C" fn __bswapsi2(x: u32) -> u32 {
        ((x & 0x000000FFu32) << 24) |
        ((x & 0x0000FF00u32) <<  8) |
        ((x & 0x00FF0000u32) >>  8) |
        ((x & 0xFF000000u32) >> 24)
    }
    pub extern "C" fn __bswapdi2(x: u64) -> u64 {
        ((x & 0x00000000000000FFu64) << 56) |
        ((x & 0x000000000000FF00u64) << 40) |
        ((x & 0x0000000000FF0000u64) << 24) |
        ((x & 0x00000000FF000000u64) <<  8) |
        ((x & 0x000000FF00000000u64) >>  8) |
        ((x & 0x0000FF0000000000u64) >> 24) |
        ((x & 0x00FF000000000000u64) >> 40) |
        ((x & 0xFF00000000000000u64) >> 56)
    }
    pub extern "C" fn __bswapti2(x: u128) -> u128 {
        let hi = ((x >> 64) & 0xFFFFFFFFFFFFFFFFu128) as u64;
        let lo = (x & 0xFFFFFFFFFFFFFFFFu128) as u64;
        let hi_swapped = __bswapdi2(hi) as u128;
        let lo_swapped = __bswapdi2(lo) as u128;
        (lo_swapped << 64) | hi_swapped
    }
}
BSWAP_EOF
    echo "Modified: $f"
done

# Clear compilation cache to trigger recompilation
find ~/riscion/sw/rust/testfmt/target -name "*compiler_builtins*" -delete
After modification, simply re-run run.sh. This fix must be reapplied after upgrading the nightly version.

Impact Scope

All code using core::fmt with alternate/alignment/padding formatting flags, including but not limited to:

{:#x}, {:#X}, {:#o}, {:#b}

{:#010x} and other width-specified alternate formats

{:>10}, {:<10}, {:^10} and other alignment formats

{:*^12}, {:0>8} and other custom padding formats

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with compiler-builtins/src/int/bswap.rs in the three paths listed, then reproduce the formatting cases through run.sh and inspect the generated call chain. Verify that the bswap implementations no longer recurse under rustc_codegen_gcc and that the alternate, alignment, and padding examples complete without hanging.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.