hellas-ai / hellas-ai/catgrad

Make sure TopK is deterministic when needed

Open
#312 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
34
Forks
7
PR merge metrics
No merged PRs in 30d

Description

The TopK operation relies on the Candle backend's asort() method which is unstable due to using partial_cmp().
This means inputs containing NaN/Inf may be sorted nondeterministically. While in MoE router layers TopK should only get regular float valued logits, we should consider reimplementing it with stable sort if/when this becomes necessary.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.