Lightning-AI / Lightning-AI/litgpt

Issues Converting lit_model.pth to Huggingface Format Using convert_from_litgpt

Open
#1,847 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted question
Dominant language
Python
Stars
13.7k
Forks
1.5k
Avg merge
15h 37m
Merged PRs (30d)
1

Description

Hello,

I have been using **litgpt** to pretrain a model, which produces a `lit_model.pth` file. This model functions correctly when loaded with `LLM.load()` for inference.

However, when I attempt to convert this model to the Huggingface format using the `convert_from_litgpt` script provided by litgpt, it outputs a `model.pth` file. This file doesn't meet Huggingface's expected format, and when I try to load it using Huggingface's tools, I receive the following error:
```python
state_dict = torch.load("model.pth")
model = AutoModel.from_pretrained(
model_id, local_files_only=True, state_dict=state_dict
)
```

```bash
OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory
```

I am unsure how to resolve this issue. Is there a step I'm missing in the conversion process, or is there a compatibility issue with the `convert_from_litgpt` script?

Additionally, I noticed that the `.pth` file obtained after training with lit-gpt is twice the size of the original model. Could you please explain why this is happening?

Thank you for your assistance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the conversion with the mentioned `lit_model.pth` and `convert_from_litgpt` script, then inspect the resulting `model.pth` against the Hugging Face loading snippet and reported filename error. Determine whether the conversion output is accepted by Hugging Face and document or correct the handling of the converted file; also investigate why the post-training `.pth` file is twice the original size.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.