Lightning-AI / Lightning-AI/lightning-thunder
NVFuser kernels: floating point scalars interpreted as `Double` and not `Float`
@kevinstephano is already working on this.
Since Nov 12, 2024.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
We have primitives that are inconsistent when it comes to interpreting input floating point scalar values.
For example, take a look at `where` (relevant issue https://github.com/Lightning-AI/lightning-thunder/issues/833), where we can see a result-meta mismatch.
Also, take a look at the fused section from https://github.com/Lightning-AI/lightning-thunder/issues/833:
```python
def nvfuser_fusion_id0(fd : FusionDefinition) -> None :
T0 = fd.define_tensor(shape=[-1, -1], contiguity=[True, True], dtype=DataType.Float, is_cpu=False, stride_order=[1, 0])
S1 = fd.define_scalar(0.00000, dtype=DataType.Double)
T2 = fd.ops.gt(T0, S1)
S3 = fd.define_scalar(-0.00000, dtype=DataType.Double)
S4 = fd.define_scalar(0.00000, dtype=DataType.Double)
T5 = fd.ops.where(T2, S3, S4)
fd.add_output(T5)
```
So, we should normalize scalar interpretation across executors.
cc @tfogal
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.