Lightning-AI / Lightning-AI/litgpt
Make `save_hyperparameters()` robust against different CLI entry points
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 1.5k
- Avg merge
- 15h 37m
- Merged PRs (30d)
- 1
Description
If you run with
```
litgpt finetune ...
```
and when getting to saving a checkpoint, we hit this line:
https://github.com/Lightning-AI/litgpt/blob/f951f9334610da35c7ecaa7e26e7ba3ac2504dab/litgpt/finetune/lora.py#L193
which reruns the CLI and parses the args that were passed. But this no longer works because it's not the same parser.
```
Saving LoRA weights to 'out/finetune/lora-llama2-7b/step-000200/lit_model.pth.lora'
usage: litgpt [-h] [--config CONFIG] [--print_config[=flags]] [--precision PRECISION] [--quantize QUANTIZE] [--devices DEVICES] [--seed SEED] [--lora_r LORA_R]
[--lora_alpha LORA_ALPHA] [--lora_dropout LORA_DROPOUT] [--lora_query {true,false}] [--lora_key {true,false}] [--lora_value {true,false}]
[--lora_projection {true,false}] [--lora_mlp {true,false}] [--lora_head {true,false}] [--data.help CLASS_PATH_OR_NAME] [--data DATA]
[--checkpoint_dir CHECKPOINT_DIR] [--out_dir OUT_DIR] [--logger_name {wandb,tensorboard,csv}] [--train CONFIG] [--train.save_interval SAVE_INTERVAL]
[--train.log_interval LOG_INTERVAL] [--train.global_batch_size GLOBAL_BATCH_SIZE] [--train.micro_batch_size MICRO_BATCH_SIZE]
[--train.lr_warmup_steps LR_WARMUP_STEPS] [--train.epochs EPOCHS] [--train.max_tokens MAX_TOKENS] [--train.max_steps MAX_STEPS]
[--train.max_seq_length MAX_SEQ_LENGTH] [--train.tie_embeddings {true,false,null}] [--train.learning_rate LEARNING_RATE]
[--train.weight_decay WEIGHT_DECAY] [--train.beta1 BETA1] [--train.beta2 BETA2] [--train.max_norm MAX_NORM] [--train.min_lr MIN_LR] [--eval CONFIG]
[--eval.interval INTERVAL] [--eval.max_new_tokens MAX_NEW_TOKENS] [--eval.max_iters MAX_ITERS]
error: Unrecognized arguments: finetune lora
```
A initial hack to fix this was done in #1103.
Comment by @carmocca
https://github.com/Lightning-AI/litgpt/pull/1103#discussion_r1523182612
> How do you think this could be done? Do we need to choose between jsonargparse.CLI or the CLI in __main__ and then pass the correct one to capture_parser?
>
> We could also simplify this by not having a CLI in the scripts themselves.
We need to make it more robust.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with `litgpt finetune ...` and inspect `litgpt/finetune/lora.py` around line 193, along with the CLI entry points discussed in #1103. Compare how each entry point parses arguments and make checkpoint saving work without reporting `finetune lora` as unrecognized; verify the command completes successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100