agentscope-ai / agentscope-ai/TuFT

Installer accepts cu129 but silently continues when its FlashAttention wheel is unavailable

未關閉
#161 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
69
分支
14
平均合併
6 小時 31 分鐘
30 天內合併 PR
13

描述

## Summary

TuFT's installer accepts an explicit `--torch-backend cu129` override, but the bundled
FlashAttention installer constructs a CUDA 12 wheel URL that does not exist for the resolved
Torch 2.11 / Python 3.12 tuple. Because FlashAttention installation is optional, the overall
installation continues successfully. A TuFT server can then pass its health check and fail much
later, during lazy FSDP model creation, when a model config explicitly requests
`attn_implementation: flash_attention_2`.

The default/supported cu130 path works; this report is about the behavior and diagnostics for the
accepted but unvalidated cu129 override.

## Environment

- TuFT revision: `102eefd4cf8fb0241ba656d14df4928f0b2c0c9e`
- Linux x86_64
- Python: 3.12.13 (`cp312`)
- PyTorch: `2.11.0+cu129`
- `torch.version.cuda`: `12.9`
- CXX11 ABI: `TRUE`
- GPU: NVIDIA A100-SXM4-80GB (SM80)
- Driver: 580.95.05

## Reproduction / observed behavior

For this tuple, `scripts/install_flash_attn.py` constructs:

```
https://dail-wlcb.oss-cn-wulanchabu.aliyuncs.com/AgentScope/download/flash-attn/2.8.1/flash_attn-2.8.1%2Bcu12torch2.11cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
```

That object returns HTTP 404 (`NoSuchKey`). The installer treats the FlashAttention failure as
optional and continues. The resulting environment has no importable `flash_attn`.

Starting TuFT itself succeeds. Later, creating a Qwen3.5 FSDP training model with:

```yaml
attn_implementation: flash_attention_2
```

fails with:

```
ImportError: FlashAttention2 has been toggled on, but ... the package for FlashAttention2 doesn't seem to be installed.
```

For comparison, the supported cu130 tuple on the same host resolves to PyTorch `2.11.0+cu130`
and successfully installs/imports `flash_attn==2.8.3`.

## Suggested improvements

Any combination of the following would prevent the late failure:

1. Use an explicit compatibility/availability table for hosted CUDA 12 wheels rather than
constructing URLs for tuples that may not exist.
2. If an explicit backend has no hosted FlashAttention wheel, print a prominent final capability
warning that includes the resolved tuple and recommends cu130 or `sdpa`.
3. When a model configuration explicitly requests `flash_attention_2`, validate that `flash_attn`
imports during configuration/server startup instead of deferring the error until lazy model
creation.
4. If cu129 is intended to be supported with FlashAttention, publish the missing tuple; otherwise
document that the accepted override does not include that capability.

Related but different: #70 reports a virtual-environment targeting failure after a wheel was
downloaded. This report concerns a missing hosted wheel and the resulting late capability failure.

貢獻指南

這個儲存庫沒有索引到貢獻指南

研究方向

Reproduce the cu129 case from the issue and inspect scripts/install_flash_attn.py, including the generated wheel URL and optional-failure handling. Trace the server startup and model-configuration path for attn_implementation: flash_attention_2, then compare the cu129 and cu130 outcomes. Done should mean the unsupported capability is surfaced before lazy model creation, or the accepted support/documentation is made explicit.

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python, pytorch
領域
build-system, machine-learning
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
活躍
描述清晰度
基本清楚
新手友好度
48/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。