alibaba / alibaba/ChatLearn

[BUG] Resource temporarily unavailable

Open
#262 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
450
Forks
40
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
I'm using megatron to train llama2-7b's rlhf alignment with 8 H100s, 60-core CPU, 1200g memory and got this error: Resource temporarily unavailable

**Screenshots**
If applicable, add screenshots to help explain your problem.

error info
![Image](https://github.com/user-attachments/assets/118fa7fd-cfaa-4ab3-a70e-b44526145dde)

![Image](https://github.com/user-attachments/assets/a3184870-6e8c-42f6-a301-9f93195851d9)

gpu use info

![Image](https://github.com/user-attachments/assets/fa846350-7d1d-46bd-80ed-84589691191b)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the two error screenshots and the reported training setup: Llama2-7B RLHF alignment with 8 H100 GPUs, 60 CPU cores, and 1200 GB memory. Reproduce the failure if possible and trace which component emits “Resource temporarily unavailable”; done means identifying a verified cause and documenting or fixing it.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.