huggingface / huggingface/candle
[Tracking] FLUX T5 XXL model produces NaN when on CUDA and using F16
Open
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
Perhaps we can use clamping, as per:
> https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py#L748-L755
Using BF16 works on CUDA.
Contributor guide
No contributing guide indexed for this repository
Research direction
Read the referenced modeling_t5.py section in Transformers and trace the FLUX T5 XXL CUDA path in this repository. Reproduce the model with F16 on CUDA, compare the existing BF16 behavior, and verify that the completed change no longer produces NaN values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100