agentscope-ai / agentscope-ai/TuFT
Installer accepts cu129 but silently continues when its FlashAttention wheel is unavailable
- Langage dominant
- Python
- Étoiles
- 69
- Forks
- 14
- Merge moyen
- 6 h 31 min
- PR mergées (30 j)
- 13
Description
## Summary
TuFT's installer accepts an explicit `--torch-backend cu129` override, but the bundled
FlashAttention installer constructs a CUDA 12 wheel URL that does not exist for the resolved
Torch 2.11 / Python 3.12 tuple. Because FlashAttention installation is optional, the overall
installation continues successfully. A TuFT server can then pass its health check and fail much
later, during lazy FSDP model creation, when a model config explicitly requests
`attn_implementation: flash_attention_2`.
The default/supported cu130 path works; this report is about the behavior and diagnostics for the
accepted but unvalidated cu129 override.
## Environment
- TuFT revision: `102eefd4cf8fb0241ba656d14df4928f0b2c0c9e`
- Linux x86_64
- Python: 3.12.13 (`cp312`)
- PyTorch: `2.11.0+cu129`
- `torch.version.cuda`: `12.9`
- CXX11 ABI: `TRUE`
- GPU: NVIDIA A100-SXM4-80GB (SM80)
- Driver: 580.95.05
## Reproduction / observed behavior
For this tuple, `scripts/install_flash_attn.py` constructs:
```
https://dail-wlcb.oss-cn-wulanchabu.aliyuncs.com/AgentScope/download/flash-attn/2.8.1/flash_attn-2.8.1%2Bcu12torch2.11cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
```
That object returns HTTP 404 (`NoSuchKey`). The installer treats the FlashAttention failure as
optional and continues. The resulting environment has no importable `flash_attn`.
Starting TuFT itself succeeds. Later, creating a Qwen3.5 FSDP training model with:
```yaml
attn_implementation: flash_attention_2
```
fails with:
```
ImportError: FlashAttention2 has been toggled on, but ... the package for FlashAttention2 doesn't seem to be installed.
```
For comparison, the supported cu130 tuple on the same host resolves to PyTorch `2.11.0+cu130`
and successfully installs/imports `flash_attn==2.8.3`.
## Suggested improvements
Any combination of the following would prevent the late failure:
1. Use an explicit compatibility/availability table for hosted CUDA 12 wheels rather than
constructing URLs for tuples that may not exist.
2. If an explicit backend has no hosted FlashAttention wheel, print a prominent final capability
warning that includes the resolved tuple and recommends cu130 or `sdpa`.
3. When a model configuration explicitly requests `flash_attention_2`, validate that `flash_attn`
imports during configuration/server startup instead of deferring the error until lazy model
creation.
4. If cu129 is intended to be supported with FlashAttention, publish the missing tuple; otherwise
document that the accepted override does not include that capability.
Related but different: #70 reports a virtual-environment targeting failure after a wheel was
downloaded. This report concerns a missing hosted wheel and the resulting late capability failure.
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.