[Question] Using Temperature and Hard Negative Mining for Retrieval Models

I've read in a couple of papers that mention that careful tuning the temperature hyperparameter is critical for retrieval quality

Quoting from the paper

From what i understand about the temperature, setting it to >1 specially in encoder decoder setting is done after the training, to make the output probability distribution become more uniformly distributed, to have more diverse outputs, here i found the temperature is used within the training and usually set from papers experiments to a number below 1 --> ~0.2

My question here is after setting it to 0.2 we're actually making the model become more confident in its prediction so the gradient magnitude is becoming smaller, so the changes aren't that dramatic due to the softmax loss function (only when the true label is predicted, but if it's not the case it has more dramatic changes, as the true output will be pushed to 0 )

Sample
True Labels = [0, 1, 0]
Logits = [9, 0.5, 3]
Softmax = [0.99, 0.0002, 0.002]
Temp = 0.2
Logits after Temp = [45, 2.5, 15]
Softmax after temp = [1.0, 3e^-19, 9e^-14]

On the other end, if it's > 1
I'm not sure what would be the intuition then, but we'll be damping the softmax for our prediction even if it's true,

In that case compared to the true labels, the gradients magnitude will become so big, so not sure about the intuition behind that and how it can be tuned (when i tried tuning the temperature, i saw warnings that some divisions encountered zero)

Also I'm not sure how hard negatives would be useful in that case?

Have anyone managed to successfully tune it and got good accuracy out of it? what is the range you have tried?

No repository files, tests, or entry points are identified. Start with the linked paper and the issue's questions about temperature, gradients, and hard negatives; done would require a clear, maintainer-supported explanation of the expected training behavior and tuning guidance.