[distributed][_tools] AssertionError: Roofline estimation needs to access CUDA capabilities to make estimations
- Dominant language
- Python
- Stars
- 113
- Forks
- 129
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 112
Description
### 🐛 Describe the bug
please get wheels from https://github.com/intel/torch-xpu-ops/actions/runs/19160533058 or use gh download
```
gh run download 19160533058 --repo intel/torch-xpu-ops --name Torch-XPU-Wheel-1826 --dir path --pattern "*.zip"
git clone -b distributed_2.10 https://github.com/daisyden/pytorch.git
cd pytorch
pip install -r requirements.txt
pip install pytest expecttest
pytest -v test/distributed/_tools/test_sac_ilp.py::TestSACILP::test_sac_ilp_case1
pytest -v test/distributed/_tools/test_sac_ilp.py::TestSACILP::test_sac_ilp_case2
pytest -v test/distributed/_tools/test_sac_ilp.py::TestSACILP::test_sac_ilp_case3
```
```
Traceback (most recent call last):
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/unittest/case.py", line 59, in testPartExecutor
yield
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/unittest/case.py", line 591, in run
self._callTestMethod(testMethod)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/unittest/case.py", line 549, in _callTestMethod
method()
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 3342, in wrapper
method(*args, **kwargs)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1749, in wrapper
fn(*args, **kwargs)
File "/home/sdp/xiangdong/pytorch/test/distributed/_tools/test_sac_ilp.py", line 204, in test_sac_ilp_case3
mod_info = self._collect_module_info_with_fake_tensor_mode()
File "/home/sdp/xiangdong/pytorch/test/distributed/_tools/test_sac_ilp.py", line 125, in _collect_module_info_with_fake_tensor_mode
runtime_estimator = self._run_and_get_runtime_estimator(
File "/home/sdp/xiangdong/pytorch/test/distributed/_tools/test_sac_ilp.py", line 107, in _run_and_get_runtime_estimator
_run_one_step() # We use only one iteration for estimation
File "/home/sdp/xiangdong/pytorch/test/distributed/_tools/test_sac_ilp.py", line 97, in _run_one_step
output = model(inp)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1783, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1889, in _call_impl
return inner()
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1837, in inner
result = forward_call(*args, **kwargs)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/testing/_internal/distributed/_tensor/common_dtensor.py", line 224, in forward
h = self.tok_embeddings(tokens)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1783, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1889, in _call_impl
return inner()
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1837, in inner
result = forward_call(*args, **kwargs)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 192, in forward
return F.embedding(
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/nn/functional.py", line 2567, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/distributed/_tools/runtime_estimator.py", line 455, in __torch_dispatch__
res, op_time = self._estimate(func, args, kwargs)
File "/home/sdp/miniforge-pypy3/envs/xccl_ww27/lib/python3.10/site-packages/torch/distributed/_tools/runtime_estimator.py", line 296, in _roofline_estimate
assert torch.cuda.is_available(), (
AssertionError: Roofline estimation needs to access CUDA capabilities to make estimations
```
### Versions
https://github.com/daisyden/pytorch/tree/distributed_2.10
Contributor guide
Assessment
This issue has not been assessed yet.