Lightning-AI / Lightning-AI/lightning-thunder
Support symbolic integer scalars
@kshitij12345 is already working on this.
Since Dec 11, 2025.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
Thunder JIT needs to support symbolic integer scalar values similar to PyTorch's Dynamo, which uses `SymInt` type annotations. This is critical for handling dynamic shapes and values that are only known at runtime.
## Current Behavior
Thunder likely treats all integer inputs as concrete values, causing recompilation for each new integer value.
## Expected Behavior
Thunder should support symbolic integer scalars that can represent runtime values, enabling:
- Symbolic type annotations in traces (e.g., `i0: SymbolicInt`)
- Tracking symbolic values through computation
- Proper code generation for dynamic values
## Motivation
- Pattern appears in KV-cache management for HF transformers
## Minimal Reproduction Case
```py
import torch
import thunder
@thunder.jit
def update_cumulative_length(cumulative_length: int, seq_len: int = 1) -> int:
"""cumulative_length should be treated as symbolic."""
return cumulative_length + seq_len
cumulative_length = 1024
new_length = update_cumulative_length(cumulative_length)
assert new_length == 1025
# With torch.compile(dynamic=True), cumulative_length becomes Sym(variable_name)
# Thunder should support similar symbolic tracking without concretizing values
print(update_cumulative_length._lc_cs.last_epilogue_traces[-1])
# def epilogue():
# return 1025
```
## Success Criteria
- [ ] Thunder can accept and track symbolic integer scalars
- [ ] Symbolic values maintain their symbolic nature through function calls
- [ ] No recompilation is happening on new values
- [ ] Proper type annotations for symbolic scalars
- [ ] Documentation on how to declare symbolic scalars
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.