Can I call `backward()` several times before calling `optim.step()` with AMP?

I'm sorry for asking here, but I'm not sure which is the preferred location for Q&A regarding apex.

Sometimes when batches that fit in GPU memory are not big enough to keep gradient variance at reasonable level, it is possible to call loss.backward() on several consecutive batches (therefore averaging the batch gradients) before executing a single step of the optimizer. My question is whether I can do the same when training with amp. The reason why I am not sure is that amp.scale_loss takes both the loss and optimizer and I am not exactly sure what are the implications of its operation on the optimizer - are all operations it does on the optimizer "averageable" or will the second run of backward incorrectly overwrite some state set in the first run?

Start by reviewing the AMP usage around amp.scale_loss, loss.backward(), and optim.step(). Determine whether repeated backward calls before one optimizer step are supported, then document the expected behavior and any optimizer-state implications.