google-research / google-research/t5x

how to decide on how many steps to train the model given a custom dataset.

Open
#609 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3k
Forks
338
PR merge metrics
No merged PRs in 30d

Description

I have been pretraining a T5_1_1 base model locally on GPUs. i have currently reduced the batch_size from 256 to 64 to support training on my GPUs. However i needed some advice on how to decide on how many steps should i train my model, given that i am training on a custom dataset and i have reduced the batch_size.

currently after going through the documentation, i am using the default 1000000 steps to train my model.

Contributor guide

Open the contributing guide

Research direction

The issue asks for training-step guidance for a custom dataset and a reduced batch size, but names no file, test, or entry point. Start by reviewing the existing training documentation and configuration guidance; a useful outcome would need documented criteria for selecting steps, including how the batch-size change affects that choice.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.