deepspeedai / deepspeedai/DeepSpeed

[BUG] `import deepspeed` crashes on `deepspeed==0.16.3` with `triton==3.2.0` on CPU machine

Open
#7,028 6 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug training
Dominant language
Python
Stars
43.1k
Forks
5k
Avg merge
4d 15h
Merged PRs (30d)
112

Description

Describe the bug
A clear and concise description of what the bug is.

Before this updates of triton, it's safe to import deepspeed on a CPU machine.

To Reproduce
run import deepspeed on a CPU machine will lead to the following error message:

>>> import deepspeed
[2025-02-12 18:28:06,516] [WARNING] [real_accelerator.py:181:get_accelerator] Setting accelerator to CPU. If you have GPU or other accelerator, we were unable to detect it.
[2025-02-12 18:28:06,530] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cpu (auto detect)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/__init__.py", line 25, in <module>
    from . import ops
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/__init__.py", line 11, in <module>
    from . import transformer
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/__init__.py", line 7, in <module>
    from .inference.config import DeepSpeedInferenceConfig
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/__init__.py", line 7, in <module>
    from ....model_implementations.transformers.ds_transformer import DeepSpeedTransformerInference
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/model_implementations/__init__.py", line 6, in <module>
    from .transformers.ds_transformer import DeepSpeedTransformerInference
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/model_implementations/transformers/ds_transformer.py", line 18, in <module>
    from deepspeed.ops.transformer.inference.triton.mlp import TritonMLP
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/triton/__init__.py", line 10, in <module>
    from .ops import *
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/triton/ops.py", line 6, in <module>
    import deepspeed.ops.transformer.inference.triton.matmul_ext as matmul_ext
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/triton/matmul_ext.py", line 10, in <module>
    import deepspeed.ops.transformer.inference.triton.triton_matmul_kernel as triton_matmul_kernel
  File "/home/ray/anaconda3/lib/python3.9/site-packages/deepspeed/ops/transformer/inference/triton/triton_matmul_kernel.py", line 120, in <module>
    def _fp_matmul(
  File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/autotuner.py", line 368, in decorator
    return Autotuner(fn, fn.arg_names, configs, key, reset_to_zero, restore_value, pre_hook=pre_hook,
  File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/autotuner.py", line 130, in __init__
    self.do_bench = driver.active.get_benchmarker()
  File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/driver.py", line 23, in __getattr__
    self._initialize_obj()
  File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/driver.py", line 20, in _initialize_obj
    self._obj = self._init_fn()
  File "/home/ray/anaconda3/lib/python3.9/site-packages/triton/runtime/driver.py", line 8, in _create_driver
    raise RuntimeError(f"{len(actives)} active drivers ({actives}). There should only be one.")
RuntimeError: 0 active drivers ([]). There should only be one.

Expected behavior
A clear and concise description of what you expected to happen.

import deepspeed on a CPU mahcine should not crash.

ds_report output
Please run ds_report to give us details about your setup.

Screenshots
If applicable, add screenshots to help explain your problem.

System info (please complete the following information):

  • OS: [e.g. Ubuntu 18.04]
  • GPU count and types [e.g. two machines with x8 A100s each]
  • Interconnects (if applicable) [e.g., two machines connected with 100 Gbps IB]
  • Python version
  • Any other relevant info about your setup

Launcher context
Are you launching your experiment with the deepspeed launcher, MPI, or something else?

Docker context
Are you using a specific docker image that you can share?

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing import deepspeed on a CPU machine with deepspeed==0.16.3 and triton==3.2.0, then read deepspeed/ops/transformer/inference/triton/triton_matmul_kernel.py and the import path shown in the traceback. Done means importing DeepSpeed no longer raises the zero-active-drivers error on CPU; the issue names no test file, so identify the relevant import or Triton coverage before changing it.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.