[distributed] /handler/dump_nccl_trace_pickle and nccl_log need in distributed ut tests
- Dominant language
- Python
- Stars
- 113
- Forks
- 128
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 107
Description
### 🐛 Describe the bug
```
git clone -b distributed_2.11 https://github.com/daisyden/pytorch.git
cd pytorch
pip install -r requirements.txt
pip install pytest expecttest
pytest -v test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardForceSumReduction::test_fully_shard_force_sum_both_reductions
pytest -v test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardForceSumReduction::test_fully_shard_force_sum_reduce_scatter
pytest -v test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle
pytest -v test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle_with_json
pytest -v test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle_with_params
```
```
Traceback (most recent call last):
File "/home/sdp/miniforge3/envs/xccl_test/lib/python3.10/site-packages/torch/testing/_internal/common_distributed.py", line 986, in run_test
getattr(self, test_name)()
File "/home/sdp/miniforge3/envs/xccl_test/lib/python3.10/site-packages/torch/testing/_internal/common_distributed.py", line 833, in wrapper
fn()
File "/home/sdp/miniforge3/envs/xccl_test/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 3364, in wrapper
method(*args, **kwargs)
File "/home/sdp/miniforge3/envs/xccl_test/lib/python3.10/site-packages/torch/testing/_internal/common_distributed.py", line 241, in wrapper
return func(*args, **kwargs)
File "/home/sdp/xiangdong/pytorch/test/distributed/_composable/fsdp/test_fully_shard_comm.py", line 1755, in test_fully_shard_force_sum_reduce_scatter
with open(self.nccl_log_dir.name + "/nccl_log") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp2cavipze/nccl_log'
```
### Versions
https://github.com/daisyden/pytorch/tree/distributed_2.11
Contributor guide
Assessment
This issue has not been assessed yet.