agentscope-ai / agentscope-ai/TuFT
Installer accepts cu129 but silently continues when its FlashAttention wheel is unavailable
- Linguagem predominante
- Python
- Estrelas
- 69
- Forks
- 14
- Merge médio
- 6h 31min
- PRs com merge (30d)
- 13
Descrição
## Summary
TuFT's installer accepts an explicit `--torch-backend cu129` override, but the bundled
FlashAttention installer constructs a CUDA 12 wheel URL that does not exist for the resolved
Torch 2.11 / Python 3.12 tuple. Because FlashAttention installation is optional, the overall
installation continues successfully. A TuFT server can then pass its health check and fail much
later, during lazy FSDP model creation, when a model config explicitly requests
`attn_implementation: flash_attention_2`.
The default/supported cu130 path works; this report is about the behavior and diagnostics for the
accepted but unvalidated cu129 override.
## Environment
- TuFT revision: `102eefd4cf8fb0241ba656d14df4928f0b2c0c9e`
- Linux x86_64
- Python: 3.12.13 (`cp312`)
- PyTorch: `2.11.0+cu129`
- `torch.version.cuda`: `12.9`
- CXX11 ABI: `TRUE`
- GPU: NVIDIA A100-SXM4-80GB (SM80)
- Driver: 580.95.05
## Reproduction / observed behavior
For this tuple, `scripts/install_flash_attn.py` constructs:
```
https://dail-wlcb.oss-cn-wulanchabu.aliyuncs.com/AgentScope/download/flash-attn/2.8.1/flash_attn-2.8.1%2Bcu12torch2.11cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
```
That object returns HTTP 404 (`NoSuchKey`). The installer treats the FlashAttention failure as
optional and continues. The resulting environment has no importable `flash_attn`.
Starting TuFT itself succeeds. Later, creating a Qwen3.5 FSDP training model with:
```yaml
attn_implementation: flash_attention_2
```
fails with:
```
ImportError: FlashAttention2 has been toggled on, but ... the package for FlashAttention2 doesn't seem to be installed.
```
For comparison, the supported cu130 tuple on the same host resolves to PyTorch `2.11.0+cu130`
and successfully installs/imports `flash_attn==2.8.3`.
## Suggested improvements
Any combination of the following would prevent the late failure:
1. Use an explicit compatibility/availability table for hosted CUDA 12 wheels rather than
constructing URLs for tuples that may not exist.
2. If an explicit backend has no hosted FlashAttention wheel, print a prominent final capability
warning that includes the resolved tuple and recommends cu130 or `sdpa`.
3. When a model configuration explicitly requests `flash_attention_2`, validate that `flash_attn`
imports during configuration/server startup instead of deferring the error until lazy model
creation.
4. If cu129 is intended to be supported with FlashAttention, publish the missing tuple; otherwise
document that the accepted override does not include that capability.
Related but different: #70 reports a virtual-environment targeting failure after a wheel was
downloaded. This report concerns a missing hosted wheel and the resulting late capability failure.
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Direção de pesquisa
Reproduce the cu129 case from the issue and inspect scripts/install_flash_attn.py, including the generated wheel URL and optional-failure handling. Trace the server startup and model-configuration path for attn_implementation: flash_attention_2, then compare the cu129 and cu130 outcomes. Done should mean the unsupported capability is surfaced before lazy model creation, or the accepted support/documentation is made explicit.
Escrita pelo modelo de indexação a partir do texto da issue.
Avaliação
- Stack de tecnologia
- python, pytorch
- Domínio
- build-system, machine-learning
- Tipo de issue
- Bug
- Dificuldade
- 4/5
- Tempo estimado
- 3-5 dias
- Status de atividade
- Ativa
- Clareza
- Razoavelmente clara
- Facilidade para iniciantes
- 48/100