Improve documentation of assumptions for symmetric memory
- Dominant language
- Python
- Stars
- 103k
- Forks
- 29.6k
- PR merge metrics
- PR metrics pending
Description
### 📚 The doc issue
Current docs for [symmetric memory ](https://docs.pytorch.org/docs/stable/symmetric_memory.html) specify that "The empty and rendezvous functions must be called in the same order on all ranks in the group." but IMO, this should be improved:
1. This sentence appears under the “Hello World” example section, which isn't ideal. It would be good to repeat it within the API reference or provide a link to a specific section outlining all assumptions
2. Is my understanding correct that symmetric memory also expects all ranks to provide the same `size` for `empty` ? If this is the case, it should be documented somewhere. If not, how would the strided memory look like / interact with symm_mem ops ?
3. I believe the same assumptions apply for the case where a [memory pool](https://docs.pytorch.org/docs/stable/symmetric_memory.html#using-memory-pool) is used : it would be good to add this to that section as well (maybe through a link).
While looking for this, I noticed that https://github.com/pytorch/pytorch/issues/164281 is still open but I believe it can be closed.
CC: @kwen2501
### Suggest a potential alternative/fix
_No response_
cc @svekars @sekyondaMeta @AlannaBurke
Contributor guide
Assessment
This issue has not been assessed yet.