Lightning-AI / Lightning-AI/pytorch-lightning

PTL 2.2 specifically causes torchscript errors when loaded in any environment not containing PTL 2.2

Open
#19,744 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

### Bug description

A pytorch lightning model that is converted to torchscript and saved in an environment running pytorch lightning 2.2 cannot be loaded in any environment not containing pytorch lightning 2.2, including both python environments and environments running other languages such as C++. This is obviously completely breaking, as PTL 2.2 cannot be installed in C++ environment, and this means we cannot serve our models.

Temp solution: downgrade to pytorch lightning 2.0, which does not have this problem

### What version are you seeing the problem on?

v2.2

### How to reproduce the bug
in environment running PTL 2.2:
```python

import torch

model = SomeLightningModle(...)
jit_model = model.to_torchscript()
torch.save(jit_model, "test_model.pt")
```

in any environment not running PTL 2.2, say python for example:
```
import torch

jit_model = torch.load("test_model.pt")
```
JIT models are supposed to be completely package independent. Of course only in python we could just always run PTL 2.2, but this is not an option in C++ deployment and this makes it completely breaking for that use case.
```

### Error messages and logs

```
2024-04-05 00:54:30.593
class AttributeDict:

2024-04-05 00:54:30.593
Serialized File "code/__torch__/pytorch_lightning/utilities/parsing.py", line 2
2024-04-05 00:54:30.593
expected indent but found 'newline' here:
2024-04-05 00:54:30.593
Exception in loading model z5yixmej
```

### Environment

This occurs across environments.

### More info

_No response_

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the model.to_torchscript() and torch.save/torch.load reproduction described in the issue, then inspect the serialized output and the referenced pytorch_lightning/utilities/parsing.py path. Compare artifacts produced with PTL 2.2 and 2.0 across the stated loading environments. Done means a PTL 2.2-produced TorchScript model loads without requiring PTL 2.2, including in the C++ deployment case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.