deepspeedai / deepspeedai/DeepSpeed
[BUG] `import deepspeed` crashes on `deepspeed==0.16.3` with `triton==3.2.0` on CPU machine
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43.1k
- Forks
- 5k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 112
Description
Describe the bug
A clear and concise description of what the bug is.
- deepspeed uses the @triton.autotuner decorator, which leads to the autotuner being initialized when
import deepspeedhappens - in triton 3.2.0, they add logic to the autotuner that leads to a check for torch.cuda.is_available() in the autotuner constructor
Before this updates of triton, it's safe to import deepspeed on a CPU machine.
To Reproduce
run import deepspeed on a CPU machine will lead to the following error message:
>>> import deepspeed
[2025-02-12 18:28:06,516] [WARNING] [real_accelerator.py:181:get_accelerator] Setting accelerator to CPU. If you have GPU or other accelerator, we were unable to detect it.
[2025-02-12 18:28:06,530] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cpu (auto detect)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/__init__.py", line 25, in <module>
from . import ops
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/__init__.py", line 11, in <module>
from . import transformer
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/__init__.py", line 7, in <module>
from .inference.config import DeepSpeedInferenceConfig
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/__init__.py", line 7, in <module>
from ....model_implementations.transformers.ds_transformer import DeepSpeedTransformerInference
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/model_implementations/__init__.py", line 6, in <module>
from .transformers.ds_transformer import DeepSpeedTransformerInference
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/model_implementations/transformers/ds_transformer.py", line 18, in <module>
from deepspeed.ops.transformer.inference.triton.mlp import TritonMLP
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/triton/__init__.py", line 10, in <module>
from .ops import *
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/triton/ops.py", line 6, in <module>
import deepspeed.ops.transformer.inference.triton.matmul_ext as matmul_ext
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/triton/matmul_ext.py", line 10, in <module>
import deepspeed.ops.transformer.inference.triton.triton_matmul_kernel as triton_matmul_kernel
File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/triton/triton_matmul_kernel.py", line 120, in <module>
def _fp_matmul(
File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/autotuner.py", line 368, in decorator
return Autotuner(fn, fn.arg_names, configs, key, reset_to_zero, restore_value, pre_hook=pre_hook,
File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/autotuner.py", line 130, in __init__
self.do_bench = driver.active.get_benchmarker()
File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/driver.py", line 23, in __getattr__
self._initialize_obj()
File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/driver.py", line 20, in _initialize_obj
self._obj = self._init_fn()
File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/driver.py", line 8, in _create_driver
raise RuntimeError(f"{len(actives)} active drivers ({actives}). There should only be one.")
RuntimeError: 0 active drivers ([]). There should only be one.
Expected behavior
A clear and concise description of what you expected to happen.
import deepspeed on a CPU mahcine should not crash.
ds_report output
Please run ds_report to give us details about your setup.
Screenshots
If applicable, add screenshots to help explain your problem.
System info (please complete the following information):
- OS: [e.g. Ubuntu 18.04]
- GPU count and types [e.g. two machines with x8 A100s each]
- Interconnects (if applicable) [e.g., two machines connected with 100 Gbps IB]
- Python version
- Any other relevant info about your setup
Launcher context
Are you launching your experiment with the deepspeed launcher, MPI, or something else?
Docker context
Are you using a specific docker image that you can share?
Additional context
Add any other context about the problem here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing import deepspeed on a CPU machine with deepspeed==0.16.3 and triton==3.2.0, then read deepspeed/ops/transformer/inference/triton/triton_matmul_kernel.py and the import path shown in the traceback. Done means importing DeepSpeed no longer raises the zero-active-drivers error on CPU; the issue names no test file, so identify the relevant import or Triton coverage before changing it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100