AnswerDotAI / AnswerDotAI/fsdp_qlora
Q on comparison with SFTTrainer
- Dominant language
- Jupyter Notebook
- Stars
- 1.6k
- Forks
- 201
- PR merge metrics
- No merged PRs in 30d
Description
The README mentions:
```
The SFTTrainer version has to run with a lower batch size (4 vs 8) so we only do 2 gradient accumulation steps vs 4 in the QLoRA+FSDP version.
```
Is this reversed? If the batch size is smaller with SFTTrainer, wouldn't you use higher gradient accumulation?
Separately, I note that SFTTrainer and fsdp trainings take the same time on the graph shown. I assume SFTTrainer is using DDP, so it should be quite a bit slower, no? Perhaps even close to 2x slower because the batch size is smaller so there are more forward passes required?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the README passage comparing SFTTrainer and QLoRA+FSDP, then inspect the training settings and graph referenced there. Confirm whether the batch-size, gradient-accumulation, and timing explanations are correct; done means the README comparison is unambiguous and accurately explains any observed difference.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100