Anybody know what is the version of `flash_attn` used for finetune?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
When attempting to execute the `FastChat\scripts\train_vicuna_7b.sh` script, it raises an exception with the following error message:
```
File "/usr/local/lib/python3.10/dist-packages/transformer_engine/pytorch/transformer.py", line 16, in
from flash_attn.flash_attn_interface import flash_attn_unpadded_func
ImportError: cannot import name 'flash_attn_unpadded_func' from 'flash_attn.flash_attn_interface' (/usr/local/lib/python3.10/dist-packages/flash_attn/flash_attn_interface.py)
```
Does anyone know why this error occurred? Additionally, why hasn't the repository provided a `requirements.txt` file to specify the required environment for fine-tuning the model?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with FastChat/scripts/train_vicuna_7b.sh and reproduce the reported import failure involving transformer_engine and flash_attn.flash_attn_interface. Determine which flash_attn version is compatible with this fine-tuning path and document the required environment or dependency setup. Done means the script runs past this import and the needed requirements are clearly recorded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100