Lightning-AI / Lightning-AI/lightning-thunder
symbolic cache policy can't handle string inputs properly.
@jjsjann123 is already working on this.
Since Jan 28, 2025.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
## 🐛 Bug
```
import thunder
import torch
device = "cuda"
def foo(b):
# TODO: 'device=device' doesn't work for "symbolic values" cache policy
a = torch.arange(24, device=device).reshape(3, 8)
return a + b
jfoo = thunder.jit(foo, cache="symbolic values")
out = jfoo(5)
```
The script hits
```
File "/volume/thunder_jit.py", line 9, in foo
a = torch.arange(24, device=device).reshape(3, 8)
^^^^^^^^^^^^^^^^^^
File "/opt/pytorch/lightning-thunder/thunder/core/interpreter.py", line 1302, in wrapping_wrapper
res = ufn(*uargs, **ukwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/pytorch/lightning-thunder/thunder/core/jit_ext.py", line 374, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/opt/pytorch/lightning-thunder/thunder/core/symbol.py", line 323, in __call__
result = self.meta(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/pytorch/lightning-thunder/thunder/core/langctxs.py", line 136, in _fn
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/opt/pytorch/lightning-thunder/thunder/torch/__init__.py", line 573, in arange
device = maybe_get_default_device(device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/pytorch/lightning-thunder/thunder/torch/__init__.py", line 130, in maybe_get_default_device
return device or get_default_device()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/pytorch/lightning-thunder/thunder/core/proxies.py", line 333, in __bool__
raise NotImplementedError(f"__bool__ is not implemented for {type(self)}")
NotImplementedError: __bool__ is not implemented for
```
inline device="cuda", the issue goes away. It also only errors out with symbolic values cache, which is not surprising since the other cache would hard code the string as a constant.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.