ByteDance-Seed / ByteDance-Seed/Bagel
NCCL Error & torch Version?
Open
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
Thanks for your work! I encountered the following issues:
- When I tried to perform fine-tuning using the scripts in picture, a process allocation error occurred below.
- When I tried to downgrade Torch to version 2.4, I found that Torch 2.4 does not support flex-attn.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the fine-tuning scripts and NCCL/process-allocation error shown in the attached images, then check the project's PyTorch and flex-attn version requirements. Done means the reported fine-tuning run no longer hits the allocation error and the supported Torch version is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100