Streamline LitGPT API

Following up on a longer internal discussion we had (cc @carmocca @lantiga @awaelchli ), we want to support the following user-friendly API in LitGPT:

# ligpt [action] [model]
litgpt  download  meta-llama/Meta-Llama-3-8B-Instruct
litgpt  chat      meta-llama/Meta-Llama-3-8B-Instruct
litgpt  finetune  meta-llama/Meta-Llama-3-8B-Instruct
litgpt  pretrain  meta-llama/Meta-Llama-3-8B-Instruct
litgpt  serve     meta-llama/Meta-Llama-3-8B-Instruct

So, in other words, we would make the <repo_id> a positional argument.

 

1) Root dir

We probably should introduce a --checkpoint_root_dir defaulting to "checkpoints" so that

litgpt download <repo_id>

downloads to checkpoints/<repo_id>

and

litgpt finetune <repo_id>

uses checkpoints/<repo_id>

 

2) Pretrain behavior

In pretrain, if someone runs

litgpt pretrain <repo_id>

the question is whether the <repo_id> takes the place of --model_name or --initial_checkpoint_dir. I don't have a strong opinion here, but pretraining from scratch would probably be the first thing someone thinks of when reading pretrain (as opposed to continued pretraining).

In this case we would use meta-llama/Meta-Llama-3-8B-Instruct as --model_name (or extract Meta-Llama-3-8B-Instruct as model name`).

The counter argument is that using this positional argument to replace initial_checkpoint_dir would be more consistent with litgpt download and litgpt finetune etc.

 

3) Removing subcommands

This means that we probably cannot support subcommands as in

litgpt finetune <repo_id> and lora finetune lora <repo_id>

(Please correct me if I'm wrong.)

So, we would change it to

lora finetune <repo_id> --method "lora"

 

4) Deprecating --repo_id and --checkpoint_dir

Maybe a full deprecation would not be possible due to the subcommands issue above, but we should probably at least keep --repo_id and --checkpoint_dir for now, and if they are not empty strings, we quit with a message explaining the API change (instead of a error stack trace).