dubious `&raw const|mut value` compile error

Code:

fn minimal() {
    let mut target = 10;
    let mutref = &mut target;
    let ptr = &raw mut target; // can also be &raw const target;
    drop(mutref)
}

According to the reference, the &raw mut|const val syntax should be used "whenever creating a reference would introduce incorrect aliasing assumptions". We also know that pointers acting as funny usize's is safe as long as we don't read from/write to them(or invoke other unsafe functions or unsafe superpowers with them) [citation needed]. The compiler error we get from this contradicts this:

error[E0499]: cannot borrow `target` as mutable more than once at a time
 --> src/lib.rs:4:15
  |
3 |     let mutref = &mut target;
  |                  ----------- first mutable borrow occurs here
4 |     let ptr = &raw mut target;
  |               ^^^^^^^^^^^^^^^ second mutable borrow occurs here
5 |     drop(mutref)
  |          ------ first borrow later used here

For more information about this error, try `rustc --explain E0499`.
warning: `sdfvkj` (lib) generated 1 warning
error: could not compile `sdfvkj` (lib) due to 1 previous error; 1 warning emitted

The whole premise of the addr_of! and addr_of_mut! macros and the &raw mut|const value syntax is to not go through the intermediary &T or &mut T where we can't guarantee the soundness of the existence of such references.

The actual real life scenario that caused this problem stems from using the pointer as an indicator of which value is being pointed to:

let mut val1 = ...;
let mut val2 = ...;

let ref1 = &mut val1;
let ref2 = &mut val2;

loop {
    (ref1, ref2) = (ref2, ref1); // just a normal swap

    let ptr = &raw const val1; // error!
    if std::ptr::eq(ptr, ref1) {
        // ref1 points to val1
    } else {
        // ref1 points to val2
    }
}

Which doesn't use the pointer to read from/write to the value at all, just an indicator to which stack value is being pointed.

Start with the minimal examples and the Reference section on raw borrow operators linked in the issue; compare the borrow checker's treatment of &raw const and &raw mut with the documented aliasing purpose. Determine whether the reported errors are incorrect, and consider how the val1/val2 pointer-comparison scenario should be covered by a regression test; done means the intended non-dereferencing use compiles or is clearly documented as invalid.