deepspeedai / deepspeedai/DeepSpeed

EvoformerAttention should auto-detect CUTLASS instead of requiring CUTLASS_PATH

Open
#8,001 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
43.1k
Forks
5k
Avg merge
4d 15h
Merged PRs (30d)
112

Description

Problem

DS4Sci_EvoformerAttention depends on CUTLASS, but the current builder requires users to set CUTLASS_PATH manually before the op is considered compatible. In a normal checkout with no CUTLASS_PATH, the builder currently reports:

[WARNING] Please specify CUTLASS location directory as environment variable CUTLASS_PATH
[WARNING] Possible values are: a path, DS_IGNORE_CUTLASS_DETECTION and DS_USE_CUTLASS_PYTHON_BINDINGS
is_compatible = False

That makes the setup path more fragile than necessary for editable installs, CI images, package-managed environments, and systems where CUTLASS is already installed in a conventional location.

Proposed solution

Add automatic CUTLASS discovery to the EvoformerAttention op builder while preserving CUTLASS_PATH as the explicit override.

The proposed implementation follows PyTorch's CUDA discovery pattern in torch.utils.cpp_extension: honor the explicit environment variable first, then infer from installed packages and conventional filesystem locations, and finally fail with an actionable message if discovery cannot succeed.

A proposed implementation is available here:

https://github.com/deepspeedai/DeepSpeed/compare/master...MaxTretikov:DeepSpeed:master?expand=1

Why this is valuable

This improves first-run usability without removing any existing escape hatches. Users who already set CUTLASS_PATH keep the same behavior, while users with nvidia-cutlass, a local cutlass checkout, or a system installation can build EvoformerAttention without extra environment configuration.

It also makes DeepSpeed's extension build behavior more consistent with the broader CUDA/PyTorch ecosystem. CUDA is not treated as "always manually set CUDA_HOME"; PyTorch attempts discovery first and uses the environment variable as an override. CUTLASS should follow the same principle for this op.

Why DeepSpeed should already support this

EvoformerAttention is shipped as part of DeepSpeed's op-builder system, and CUTLASS is a required build dependency for that op. Requiring every user to manually export a path for a dependency that can be discovered predictably is unnecessary friction.

Automatic discovery is additive and backwards-compatible: explicit CUTLASS_PATH remains authoritative, DS_IGNORE_CUTLASS_DETECTION still works for specialized build environments, and failure cases produce clearer guidance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the DeepSpeed EvoformerAttention op builder and compare its compatibility checks with PyTorch's torch.utils.cpp_extension CUDA discovery pattern. Preserve CUTLASS_PATH as the override, retain DS_IGNORE_CUTLASS_DETECTION behavior, and verify that installed-package and conventional-location cases either discover CUTLASS or produce actionable guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
build-system, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.