Training lasts just 150 seconds for TinyLlama OpenWebtext dataset

Nobody has claimed this yet.

Hello

I am trying to train a model on openwebtext dataset and TinyLlama model, as specified here ([https://github.com/Lightning-AI/litgpt/blob/main/tutorials/0_to_litgpt.md#pretrain-llms](https://github.com/Lightning-AI/litgpt/blob/main/tutorials/0_to_litgpt.md#pretrain-llms)) but the training lasts for just 150 seconds.

I am not sure where does it gets it's configuration from or how do i make sure the training runs longer for a better model. Can anyone help me out with that?

Command used

`CUDA_VISIBLE_DEVICES=0,1,2 TORCHDYNAMO_DISABLE=1 TORCHDYNAMO_SUPPRESS_ERRORS=True litgpt pretrain --model_name tiny-llama-1.1b --data OpenWebText --tokenizer_dir checkpoints/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T --train.max_tokens 100000 --out_dir out/custom-model --train.global_batch_size 64 --eval.max_iters 10000 --train.micro_batch_size 2 --train.log_interval 2
`

Thanks

Start with tutorials/0_to_litgpt.md#pretrain-llms and the litgpt pretrain entry point, then reproduce the supplied command with its token, batch, and evaluation settings. Done means identifying which configuration controls the short run and verifying the expected training duration or documenting the remaining limitation.

Written by the indexing model from the issue text.