rust-lang / rust-lang/rust

Unexpected unsizing coercion introduced by unsafe block

Open
#128,880 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-MIR T-opsem T-types
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

Consider this example:

pub fn testcase() -> *const dyn Bar {
    let baz: &dyn Baz = &1;
    // Unexpected coercion in the next line
    let baz_fake = unsafe { std::mem::transmute::<_, *const dyn Bar>(baz) };
    baz_fake
}

The initial MIR after MIR building is

    bb0: {
        StorageLive(_1);
        StorageLive(_2);
        StorageLive(_3);
        StorageLive(_4);
        StorageLive(_5);
        _5 = const 1_i32;
        _4 = &_5;
        _3 = &(*_4);
        _2 = move _3 as &dyn Baz (PointerCoercion(Unsize));
        StorageDead(_3);
        FakeRead(ForLet(None), _2);
        AscribeUserType(_2, o, UserTypeProjection { base: UserType(1), projs: [] });
        StorageDead(_4);
        StorageLive(_6);
        StorageLive(_7);
        StorageLive(_8);
        _8 = _2;
        _7 = transmute::<&dyn Baz, *const dyn Bar>(move _8) -> [return: bb1, unwind: bb2];
    }

    bb1: {
        StorageDead(_8);
        _6 = move _7 as *const dyn Bar (PointerCoercion(Unsize));
        StorageDead(_7);
        FakeRead(ForLet(None), _6);
        StorageLive(_9);
        _9 = _6;
        _1 = move _9 as *const dyn Bar (PointerCoercion(Unsize));
        StorageDead(_9);
        StorageDead(_6);
        StorageDead(_5);
        StorageDead(_2);
        _0 = move _1 as *const dyn Bar (PointerCoercion(Unsize));
        StorageDead(_1);
        return;
    }

Notice the no less than three coercions in bb1, all of which coerce from *const dyn Bar to *const dyn Bar. I would have expected to see zero of them; the transmute returns exactly the type we want so there's no reason to insert a coercion.

This came up because Miri actually considers such coercions to be a simple form of upcast that can introduce UB, so unexpected coercions can lead to unexpected UB. We can just change the MIR opsem to make identity coercions true NOPs, but we don't do this for any other kind of coercion so it's an odd special case.

Cc @rust-lang/types @rust-lang/opsem

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the linked Rust Playground example in release mode and inspect the initial MIR after MIR building, especially the three identity PointerCoercion(Unsize) operations in bb1. Read the MIR coercion behavior and Miri's treatment of these coercions first; done means the transmute result is not subjected to unexpected identity coercions that can introduce unsizing-related UB.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.