Feature Request: Add torch.compile support to Flux-FP8 model.
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Expected Behavior
I am using Flux-Fp8 model and I want to add torch.compile. But it works. Only in flux-dev.safetensors model torch.compile works.
### Actual Behavior
### Steps to Reproduce
[tensor_fp8.json](https://github.com/user-attachments/files/17340285/tensor_fp8.json)
### Debug Logs
```powershell
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
CompilationError: at 8:11:
def triton_(in_ptr0, out_ptr0, xnumel, XBLOCK : tl.constexpr):
xnumel = 786432
xoffset = tl.program_id(0) * XBLOCK
xindex = xoffset + tl.arange(0, XBLOCK)[:]
xmask = xindex < xnumel
x0 = xindex
tmp0 = tl.load(in_ptr0 + (x0), None)
tmp1 = tmp0.to(tl.float32)
^
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
You can suppress this exception and fall back to eager by setting:
import torch._dynamo
torch._dynamo.config.suppress_errors = True
```
### Other
What else can I do to make the Flux model run faster? Also is it possible to run torch.compile + tensorRT? I tried and got an error.
Contributor guide
Assessment
This issue has not been assessed yet.