Lightning-AI / Lightning-AI/lightning-thunder

cover forward + loss + backward

Open
#2,186 0 comments 0 reactions 0 assignees View on GitHub
design enhancement
Dominant language
Python
Stars
1.5k
Forks
121
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature

Add a way to not only compile the model, but the `forward` + `loss` + `loss.backward` (later also `optimizer.step` + `optimizer.zero_grad`, but not here).

### Motivation

For training applications, we currently integrate with torch autograd, but we could and should aim to extend beyond this. Also, this would simplify a things, e.g. there would be no need to split the trace into forward and backward (but we need to figure out re-computation nonetheless).

### Pitch

The UX might look like this:

```python
def fn(model, inp, target):
out = model(inp)
loss = lossfn(out, target)
loss.backward()

jfn = jit(fn, models=(model,))
```

### Alternatives

Other UX possible.

### Implementation thoughts

Maybe we should figure out how to combine prologues in general. This would have applications here (how do we differentiate the model prologue from the entire function prologue) but also elsewhere.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the existing jit API and its torch autograd integration, then examine how the current trace is split between forward and backward and how prologues are handled. Done means a function containing model forward, loss computation, and loss.backward can be compiled through jit without requiring separate traces; optimizer steps are explicitly out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
compilers, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.