pytorch / pytorch/pytorch

Errors in Activation Checkpointing tutorial

Open
#176,065 0 comments 0 reactions 0 assignees View on GitHub
bot-triaged module: docs triaged
Dominant language
Python
Stars
103k
Forks
29.6k
PR merge metrics
PR metrics pending

Description

### 📚 The doc issue

Link: https://pytorch.org/blog/activation-checkpointing-techniques/
The `compute_intensive_ops` are wrong and need default appended: `aten.mm` -> `aten.mm.default`
As-is, it will silently fail to match these ops.

The linked docs specify `.default` correctly: https://docs.pytorch.org/docs/stable/checkpoint.html#torch.utils.checkpoint.create_selective_checkpoint_contexts

There is no issue tracker for the blog, so I was pointed here.

Claude wrote the following testcase for me:

```
"""Test that selective activation checkpointing requires .default (OpOverload),
not the bare OpOverloadPacket (e.g. aten.mm vs aten.mm.default).

The SAC policy function receives concrete OpOverload objects at dispatch time.
If you put OpOverloadPacket objects in your set (as the PyTorch blog example does),
the `op in expensive_ops` check silently never matches and everything is recomputed.
"""

import functools

import torch
import torch.nn as nn
from torch.utils.checkpoint import checkpoint, create_selective_checkpoint_contexts, CheckpointPolicy

aten = torch.ops.aten

def _count_decisions(policy_fn, model, x):
"""Run a forward+backward with SAC and return (must_save_count, recompute_count)."""
must_save = 0
recompute = 0

def counting_policy(ctx, op, *args, **kwargs):
nonlocal must_save, recompute
decision = policy_fn(ctx, op, *args, **kwargs)
if decision == CheckpointPolicy.MUST_SAVE:
must_save += 1
else:
recompute += 1
return decision

context_fn = functools.partial(create_selective_checkpoint_contexts, counting_policy)
out = checkpoint(model, x, use_reentrant=False, context_fn=context_fn)
out.sum().backward()
return must_save, recompute

def test_default_overload_is_required():
"""OpOverloadPacket (aten.mm) never matches; OpOverload (aten.mm.default) does."""
torch.manual_seed(0)
model = nn.Sequential(nn.Linear(32, 64), nn.ReLU(), nn.Linear(64, 16))

# --- Correct: use OpOverload (.default) ---
correct_ops = {aten.mm.default, aten.addmm.default}

def correct_policy(ctx, op, *args, **kwargs):
if op in correct_ops:
return CheckpointPolicy.MUST_SAVE
return CheckpointPolicy.PREFER_RECOMPUTE

x = torch.randn(4, 32, requires_grad=True)
saved_correct, recomputed_correct = _count_decisions(correct_policy, model, x)

# --- Wrong: use OpOverloadPacket (no .default) ---
wrong_ops = {aten.mm, aten.addmm} # these are OpOverloadPackets, not OpOverloads

def wrong_policy(ctx, op, *args, **kwargs):
if op in wrong_ops:
return CheckpointPolicy.MUST_SAVE
return CheckpointPolicy.PREFER_RECOMPUTE

x = torch.randn(4, 32, requires_grad=True)
saved_wrong, recomputed_wrong = _count_decisions(wrong_policy, model, x)

# With .default, some matmuls are saved
assert saved_correct > 0, (
f"Expected MUST_SAVE > 0 with OpOverload (.default), got {saved_correct}"
)
# Without .default, nothing matches — everything is recomputed
assert saved_wrong == 0, (
f"Expected MUST_SAVE == 0 with OpOverloadPacket (no .default), got {saved_wrong}"
)

print(f"With .default: MUST_SAVE={saved_correct}, PREFER_RECOMPUTE={recomputed_correct}")
print(f"Without .default: MUST_SAVE={saved_wrong}, PREFER_RECOMPUTE={recomputed_wrong}")

if __name__ == "__main__":
test_default_overload_is_required()
print("PASSED")
```

Output:
```
With .default: MUST_SAVE=3, PREFER_RECOMPUTE=6
Without .default: MUST_SAVE=0, PREFER_RECOMPUTE=9
PASSED
```

Also, Claude says the following should not be included, and it seems logical to me, but I did not verify it through code:
```
aten.convolution_backward,
aten._flash_attention_forward,
aten._efficient_attention_forward,
```

### Suggest a potential alternative/fix

_No response_

cc @svekars @sekyondaMeta @AlannaBurke

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.