intel / intel/torch-xpu-ops

[distributed][fsdp]test_forward_overlap_xpu AttributeError: module 'torch._C' has no attribute '_cuda_sleep'

Open
#4,463 0 comments 0 reactions 1 assignee Claimed by @githubsgi View on GitHub
bug module: distributed
Dominant language
Python
Stars
113
Forks
128
Avg merge
5d 9h
Merged PRs (30d)
112

Description

### 🐛 Describe the bug

```
pip install torch torchvision torchaudio pytorch-triton-xpu --index-url https://download.pytorch.org/whl/nightly/xpu
git clone https://github.com/pytorch/pytorch.git
cd pytorch
pip install pytest expecttest
pytest -v test/distributed/fsdp/test_fsdp_overlap.py
```
```
___________________________________ TestForwardOverlapWorldSizeOneXPU.test_forward_overlap_xpu ____________________________________
Traceback (most recent call last):
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/unittest/case.py", line 59, in testPartExecutor
yield
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/unittest/case.py", line 591, in run
self._callTestMethod(testMethod)
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/unittest/case.py", line 549, in _callTestMethod
method()
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_distributed.py", line 2186, in wrapper
raise deferred_exception
RuntimeError: Exception in worker process:
Traceback (most recent call last):
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_distributed.py", line 1946, in _worker_loop
cls._run_test_given_id(test_id)
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_distributed.py", line 1899, in _run_test_given_id
test_fn(**kwargs)
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_distributed.py", line 2189, in wrapper
fn()
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 3590, in wrapper
method(*args, **kwargs)
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 627, in instantiated_test
result = test(self, **param_kwargs)
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_distributed.py", line 326, in wrapper
return func(*args, **kwargs)
File "/home/sdp/xiangdong/pytorch/test/distributed/fsdp/test_fsdp_overlap.py", line 255, in test_forward_overlap
self._dist_train()
File "/home/sdp/xiangdong/pytorch/test/distributed/fsdp/test_fsdp_overlap.py", line 196, in _dist_train
sleep_cycles = int(100 * get_cycles_per_ms())
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 5855, in get_cycles_per_ms
vals = [measure() for _ in range(num)]
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 5855, in
vals = [measure() for _ in range(num)]
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 5837, in measure
torch.get_device_module(device)._sleep(test_cycles)
File "/home/sdp/miniforge3/envs/xpu_ao_ci/lib/python3.10/site-packages/torch/cuda/__init__.py", line 279, in _sleep
torch._C._cuda_sleep(cycles)
AttributeError: module 'torch._C' has no attribute '_cuda_sleep'
```

### Versions

https://github.com/pytorch/pytorch main branch

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.