Volatile reads and writes on aarch64 sometimes generate instructions not suitable for MMIO in protected VMs

View all comments

core::ptr::write_volatile and core::ptr::read_volatile are documented as being intended to act on I/O memory, i.e. for MMIO. These are indeed widely used by many crates providing drivers for MMIO devices across the ecosystem.

When running in a virtual machine, MMIO must be emulated by the hypervisor. This is done (on aarch64 at least) by having the MMIO region unmapped in the stage 2 page table, which results in a data abort to the hypervisor when the VM attempts to read or write the MMIO region. The hypervisor then decodes the exception syndrome register (esr_el2) and uses the fault address register (far_el2) to determine which MMIO address is being accessed and perform the appropriate operation in response.

Unfortunately, rustc sometimes compiles core::ptr::write_volatile on aarch64 to something like str w9, [x0], #4. We've seen this happen particularly since Rust 1.78, but it may be possible with earlier Rust versions too. The problem with this is that this post-addressing mode is performing register writeback (in this case, incrementing x0 by 4), and so doesn't set the exception syndrome register. This prevents the hypervisor from emulating the MMIO access, as it has no way of decoding the instruction syndrome or finding the faulting address.

In an unprotected VM (e.g. regular KVM), it is possible for the VMM to work around this by reading the guest VM's memory to find the relevant instruction, decoding the instruction manually, and finding the MMIO address that way. This has a performance overhead and adds extra complexity. In the case of a protected VM where the host doesn't have access to the guest VM's memory (e.g. protected KVM), this is not possible as the VMM is not able to read the guest VM's memory and so cannot do instruction decoding. There is thus no way to emulate these attempted MMIO accesses in a protected VM on aarch64.

The net result of this is that instructions which perform register writeback (e.g. post-increment addressing modes) are not suitable for MMIO in aarch64 VMs. This is arguably a flaw in the aarch64 architecture, but as that's not feasible to fix at this point it must be fixed in the compiler instead. rustc should therefore avoid generating such instructions for volatile_read and volatile_write calls.

The only alternative I can see to fixing this in rustc is for every crate which performs MMIO to use inline assembly rather than volatile_read / volatile_write, but that is not a very feasible or scalable solution.

Start by reproducing the AArch64 code generation for core::ptr::read_volatile and core::ptr::write_volatile, checking whether register-writeback addressing is emitted. Trace the rustc AArch64 lowering involved and add coverage for MMIO-safe instruction selection; done means volatile accesses no longer use writeback instructions on this target.