MGF-softmax, a novel softmax reformulation

https://arxiv.org/abs/2602.01621

MGF-softmax, a novel softmax reformulation based on the moment generating function (MGF) that replaces the softmax denominator with its moment-based counterpart. This reformulation substantially reduces multiplicative depth while preserving key properties of softmax and asymptotically converging to the exact softmax as the number of input tokens increases.

This issue is to recommend for someone to look through the solutions mentioned in the paper for implementation in in HEIR.