gradient_accumulation_steps integration

1. gradient_accumulation_steps configuration is not documented at all - it's only mentioned in the context of pipeline
2. there are no instructions on how to integrate it with the existing trainer - so if our trainer already handles the accumulation most likely we should reset gradient_accumulation_steps to 1 in deepspeed configuration, correct? I am asking since I don't know if perhaps something else is going on and DeepSpeed needs to know about it too - asking since I see it being used in deepspeed/runtime/zero/stage2.py.

To exemplify this is what we have in place right now:

               if (step + 1) % self.args.gradient_accumulation_steps == 0 ...:
                    # Optimizer step
                    if self.deepspeed:
                        self.deepspeed.step()
              [...]

I'm pretty sure I still need to pass gradient_accumulation_steps=N to DeepSpeed, but just asking if there is something special that we should or shouldn't do in the custom trainer.

Thank you.

Start with deepspeed/runtime/zero/stage2.py and the existing pipeline documentation to trace how gradient_accumulation_steps is consumed. Confirm the interaction with a custom trainer that already performs accumulation, then document the required DeepSpeed configuration, integration steps, and expected behavior.