Rust-GPU / Rust-GPU/rust-cuda

Data Races & Flawed Synchronization due to Invalid Atomic Memory Scope in `SystemAtomicF32` and `SystemAtomicF64

Open Beginner friendly
#399 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
5.4k
Forks
249
PR merge metrics
No merged PRs in 30d

Description

SystemAtomicF32 and SystemAtomicF64 types are mistakenly configured with device scope instead of system scope in their macro instantiations.

This presents the following issues:

  1. Flawed Synchronization: Code that uses SystemAtomicF32 or SystemAtomicF64 intends to synchronize memory accesses across the entire system (between the GPU and host CPU or across multiple GPUs over PCIe/NVLink).
  2. Data Races: Because the emitted PTX operations use the .gpu scope (atomic.global.gpu...) rather than the proper .sys scope (atomic.global.sys...), CPU host accesses and multi-GPU accesses to system memory will not observe proper cache coherency. This leads to data races and corrupted shared memory states without emitting any compilation errors.

Reproduction Case

Using SystemAtomicF32 or SystemAtomicF64 and calling atomic operations (like fetch_add, load, or store) will incorrectly generate device-scoped PTX:

use cuda_std::atomic::SystemAtomicF32;
use core::sync::atomic::Ordering;

#[cuda_std::kernel]
pub unsafe fn system_atomic_kernel(val: &SystemAtomicF32) {
    // PTX generates: atomic.global.gpu.add.f32 ...
    // Expected: atomic.global.sys.add.f32 ...
    val.fetch_add(1.0, Ordering::Relaxed);
}

Issue Details

In crates/cuda_std/src/atomic.rs, the type instantiations for system-level float atomics incorrectly pass the $scope argument as device instead of system to the macro:

// Current instantiations in atomic.rs
atomic_float!(f32, AtomicF32, 4, device, 32);
atomic_float!(f64, AtomicF64, 8, device, 64);
atomic_float!(f32, BlockAtomicF32, 4, block, 32, unsafe);
atomic_float!(f64, BlockAtomicF64, 8, block, 64, unsafe);
atomic_float!(f32, SystemAtomicF32, 4, device, 32); // <--- BUG: device instead of system
atomic_float!(f64, SystemAtomicF64, 8, device, 64); // <--- BUG: device instead of system

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in crates/cuda_std/src/atomic.rs and inspect the macro instantiations for SystemAtomicF32 and SystemAtomicF64. Change their scope argument from device to system, then verify that their atomic operations emit the expected system-scoped PTX operations instead of device-scoped operations.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.