Clarification: no attention scores normalization when using Adafactor?

_the natural scale for adafactor LR is markedly different from Adam, one doesn't use the 1/sqrt(hidden) correction for this optimizer with attention-based models._
https://github.com/google-research/t5x/blob/540a65958a1d4d60cb779771247e7312069f56b3/t5x/adafactor.py#L219

Does this mean that we should not normalize attention scores (before the softmax) when using Adafactor? If so, could you please explain why?

Start with t5x/adafactor.py at the cited line and review how the optimizer's learning-rate scale relates to attention-based models. Determine whether the issue calls for a documentation clarification, and consider the question resolved when the normalization behavior and its rationale are clearly explained.

Written by the indexing model from the issue text.