agentscope-ai / agentscope-ai/TuFT

Installer accepts cu129 but silently continues when its FlashAttention wheel is unavailable

Aberta
#161 0 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
69
Forks
14
Merge médio
6h 31min
PRs com merge (30d)
13

Descrição

## Summary

TuFT's installer accepts an explicit `--torch-backend cu129` override, but the bundled
FlashAttention installer constructs a CUDA 12 wheel URL that does not exist for the resolved
Torch 2.11 / Python 3.12 tuple. Because FlashAttention installation is optional, the overall
installation continues successfully. A TuFT server can then pass its health check and fail much
later, during lazy FSDP model creation, when a model config explicitly requests
`attn_implementation: flash_attention_2`.

The default/supported cu130 path works; this report is about the behavior and diagnostics for the
accepted but unvalidated cu129 override.

## Environment

- TuFT revision: `102eefd4cf8fb0241ba656d14df4928f0b2c0c9e`
- Linux x86_64
- Python: 3.12.13 (`cp312`)
- PyTorch: `2.11.0+cu129`
- `torch.version.cuda`: `12.9`
- CXX11 ABI: `TRUE`
- GPU: NVIDIA A100-SXM4-80GB (SM80)
- Driver: 580.95.05

## Reproduction / observed behavior

For this tuple, `scripts/install_flash_attn.py` constructs:

```
https://dail-wlcb.oss-cn-wulanchabu.aliyuncs.com/AgentScope/download/flash-attn/2.8.1/flash_attn-2.8.1%2Bcu12torch2.11cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
```

That object returns HTTP 404 (`NoSuchKey`). The installer treats the FlashAttention failure as
optional and continues. The resulting environment has no importable `flash_attn`.

Starting TuFT itself succeeds. Later, creating a Qwen3.5 FSDP training model with:

```yaml
attn_implementation: flash_attention_2
```

fails with:

```
ImportError: FlashAttention2 has been toggled on, but ... the package for FlashAttention2 doesn't seem to be installed.
```

For comparison, the supported cu130 tuple on the same host resolves to PyTorch `2.11.0+cu130`
and successfully installs/imports `flash_attn==2.8.3`.

## Suggested improvements

Any combination of the following would prevent the late failure:

1. Use an explicit compatibility/availability table for hosted CUDA 12 wheels rather than
constructing URLs for tuples that may not exist.
2. If an explicit backend has no hosted FlashAttention wheel, print a prominent final capability
warning that includes the resolved tuple and recommends cu130 or `sdpa`.
3. When a model configuration explicitly requests `flash_attention_2`, validate that `flash_attn`
imports during configuration/server startup instead of deferring the error until lazy model
creation.
4. If cu129 is intended to be supported with FlashAttention, publish the missing tuple; otherwise
document that the accepted override does not include that capability.

Related but different: #70 reports a virtual-environment targeting failure after a wheel was
downloaded. This report concerns a missing hosted wheel and the resulting late capability failure.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Direção de pesquisa

Reproduce the cu129 case from the issue and inspect scripts/install_flash_attn.py, including the generated wheel URL and optional-failure handling. Trace the server startup and model-configuration path for attn_implementation: flash_attention_2, then compare the cu129 and cu130 outcomes. Done should mean the unsupported capability is surfaced before lazy model creation, or the accepted support/documentation is made explicit.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
python, pytorch
Domínio
build-system, machine-learning
Tipo de issue
Bug
Dificuldade
4/5
Tempo estimado
3-5 dias
Status de atividade
Ativa
Clareza
Razoavelmente clara
Facilidade para iniciantes
48/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.