AnswerDotAI / AnswerDotAI/fsdp_qlora

Q on comparison with SFTTrainer

Open
#42 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.6k
Forks
201
PR merge metrics
No merged PRs in 30d

Description

The README mentions:
```
The SFTTrainer version has to run with a lower batch size (4 vs 8) so we only do 2 gradient accumulation steps vs 4 in the QLoRA+FSDP version.
```

Is this reversed? If the batch size is smaller with SFTTrainer, wouldn't you use higher gradient accumulation?

Separately, I note that SFTTrainer and fsdp trainings take the same time on the graph shown. I assume SFTTrainer is using DDP, so it should be quite a bit slower, no? Perhaps even close to 2x slower because the batch size is smaller so there are more forward passes required?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the README passage comparing SFTTrainer and QLoRA+FSDP, then inspect the training settings and graph referenced there. Confirm whether the batch-size, gradient-accumulation, and timing explanations are correct; done means the README comparison is unambiguous and accurately explains any observed difference.

Written by the indexing model from the issue text.

Assessment

Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.