deepspeedai / deepspeedai/DeepSpeedExamples
DeepSpeed-Chat Step-3 tensorboard loss figures with multiple training epochs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Description
Hi, when training RLHF step-3, I set parameters related to epochs as:
- ppo_epochs = 1
- num_train_epochs = 30
and I found that the numbers of lines in "actor_loss", "actor_loss_sum", "critic_loss", "critic_loss_sum" and "reward" (saved in the folder "step3_tensorboard_logs") are the same as the parameter "num_train_epochs", like :
The question is:
How can I make them display as the figure for "train_loss" or "lr" (saved in the folder "ds_tensorboard_logs") like the below one, where the line for the whole 30 epochs as a whole ( a single line):
Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing how logging is performed for the step-3_tensorboard_logs and ds_tensorboard_logs folders, using the ppo_epochs and num_train_epochs settings described in the issue. Determine whether the loss and reward values are logged per epoch or aggregated across all 30 epochs; done means the figures use the requested whole-training aggregation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100