google-research / google-research/t5x
how to decide on how many steps to train the model given a custom dataset.
- Dominant language
- Python
- Stars
- 3k
- Forks
- 338
- PR merge metrics
- No merged PRs in 30d
Description
I have been pretraining a T5_1_1 base model locally on GPUs. i have currently reduced the batch_size from 256 to 64 to support training on my GPUs. However i needed some advice on how to decide on how many steps should i train my model, given that i am training on a custom dataset and i have reduced the batch_size.
currently after going through the documentation, i am using the default 1000000 steps to train my model.
Contributor guide
Research direction
The issue asks for training-step guidance for a custom dataset and a reduced batch size, but names no file, test, or entry point. Start by reviewing the existing training documentation and configuration guidance; a useful outcome would need documented criteria for selecting steps, including how the batch-size change affects that choice.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100