Make sure TopK is deterministic when needed
Open
- Dominant language
- Rust
- Stars
- 34
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
The TopK operation relies on the Candle backend's asort() method which is unstable due to using partial_cmp().
This means inputs containing NaN/Inf may be sorted nondeterministically. While in MoE router layers TopK should only get regular float valued logits, we should consider reimplementing it with stable sort if/when this becomes necessary.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.