agentscope-ai / agentscope-ai/TuFT

Installer accepts cu129 but silently continues when its FlashAttention wheel is unavailable

Ouverte
#161 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
69
Forks
14
Merge moyen
6 h 31 min
PR mergées (30 j)
13

Description

## Summary

TuFT's installer accepts an explicit `--torch-backend cu129` override, but the bundled
FlashAttention installer constructs a CUDA 12 wheel URL that does not exist for the resolved
Torch 2.11 / Python 3.12 tuple. Because FlashAttention installation is optional, the overall
installation continues successfully. A TuFT server can then pass its health check and fail much
later, during lazy FSDP model creation, when a model config explicitly requests
`attn_implementation: flash_attention_2`.

The default/supported cu130 path works; this report is about the behavior and diagnostics for the
accepted but unvalidated cu129 override.

## Environment

- TuFT revision: `102eefd4cf8fb0241ba656d14df4928f0b2c0c9e`
- Linux x86_64
- Python: 3.12.13 (`cp312`)
- PyTorch: `2.11.0+cu129`
- `torch.version.cuda`: `12.9`
- CXX11 ABI: `TRUE`
- GPU: NVIDIA A100-SXM4-80GB (SM80)
- Driver: 580.95.05

## Reproduction / observed behavior

For this tuple, `scripts/install_flash_attn.py` constructs:

```
https://dail-wlcb.oss-cn-wulanchabu.aliyuncs.com/AgentScope/download/flash-attn/2.8.1/flash_attn-2.8.1%2Bcu12torch2.11cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
```

That object returns HTTP 404 (`NoSuchKey`). The installer treats the FlashAttention failure as
optional and continues. The resulting environment has no importable `flash_attn`.

Starting TuFT itself succeeds. Later, creating a Qwen3.5 FSDP training model with:

```yaml
attn_implementation: flash_attention_2
```

fails with:

```
ImportError: FlashAttention2 has been toggled on, but ... the package for FlashAttention2 doesn't seem to be installed.
```

For comparison, the supported cu130 tuple on the same host resolves to PyTorch `2.11.0+cu130`
and successfully installs/imports `flash_attn==2.8.3`.

## Suggested improvements

Any combination of the following would prevent the late failure:

1. Use an explicit compatibility/availability table for hosted CUDA 12 wheels rather than
constructing URLs for tuples that may not exist.
2. If an explicit backend has no hosted FlashAttention wheel, print a prominent final capability
warning that includes the resolved tuple and recommends cu130 or `sdpa`.
3. When a model configuration explicitly requests `flash_attention_2`, validate that `flash_attn`
imports during configuration/server startup instead of deferring the error until lazy model
creation.
4. If cu129 is intended to be supported with FlashAttention, publish the missing tuple; otherwise
document that the accepted override does not include that capability.

Related but different: #70 reports a virtual-environment targeting failure after a wheel was
downloaded. This report concerns a missing hosted wheel and the resulting late capability failure.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.