Lightning-AI / Lightning-AI/litgpt
How to set max_iters
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 1.5k
- Avg merge
- 15h 37m
- Merged PRs (30d)
- 1
Description
I am trying to pretrain a model on book corpus, and i was wondering how do i make sure the training runs for 10,000 iterations?
There a max_iters in eval, but not in train.
I am just using the documentation from the home page.
litgpt pretrain \
--model_name pythia-160m \
--tokenizer_dir checkpoints/EleutherAI/pythia-160m \
--data TextFiles \
--data.train_data_path "custom_texts/" \
--train.max_tokens 10_000_000 \
--out_dir out/custom-model
How do i pass the max iters value?
Thanks
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the homepage pretraining documentation and the litgpt pretrain entry point shown in the issue, then inspect how train.max_tokens relates to iteration limits. Document the supported way to run for 10,000 iterations, or clarify that configuration in the example and verify the command matches the documented behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100