get_device_context tensor goes stale if heap_bases change after init
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 202
- Forks
- 47
- Avg merge
- 6d 11h
- Merged PRs (30d)
- 4
Description
Bug
get_device_context() builds a new torch.tensor from self.heap_bases.tolist() on every call (see #466). Once #466 is fixed by precomputing the tensor in __init__, the context tensor will hold a snapshot of heap_bases at construction time.
If heap_bases were to change after init (e.g., via refresh_peer_access() after a new shmem.allocate() or as_symmetric() call with a future allocator), the precomputed context tensor would contain stale base addresses. Kernels using DeviceContext would translate pointers using wrong bases, causing silent data corruption or hangs.
Today this is not a bug — both the torch and vmem allocators produce stable heap_bases after the first refresh_peer_access(). But it will become one if an allocator ever remaps peer VA ranges.
Fix
After precomputing self._device_context in __init__, add an in-place update in refresh_peer_access():
self._device_context[2:2+self.num_ranks] = self.heap_bases
No allocation, CUDAGraph safe, one line.
Component
iris/iris.py, iris/symmetric_heap.py
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in iris/iris.py and inspect how the device context is initialized and refreshed, then review the related heap behavior in iris/symmetric_heap.py. Confirm that refresh_peer_access keeps the context's heap-base values current after heap_bases changes, without introducing allocations or breaking CUDA Graph safety.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100