Deepspeed activation Partitioning

Nobody has claimed this yet.

### 📚 Documentation

Hello,

- https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.strategies.DeepSpeedStrategy.html

> partition_activations ([bool](https://docs.python.org/3/library/functions.html#bool)) – Enables partition activation when used with ZeRO stage 3 and model parallelism. Still requires you to wrap your forward functions in deepspeed.checkpointing.checkpoint. See [deepspeed tutorial](https://www.deepspeed.ai/tutorials/megatron/#deepspeed-activation-checkpoints-optional).

Upon encountering issues with activation partitioning and after checking, I found that Deepspeed activation partitioning is not significantly related to the use of zero-3; rather, it appears that the setup of model parallelism and mpu object is crucial.

Also, it is explicitly stated that pipeline parallelism, a model parallelism method provided by Deepspeed, cannot be used in conjunction with zero-2 and zero-3 from the outset.
- https://deepspeed.readthedocs.io/en/latest/pipeline.html

Additionally, in the [GitHub issue referenced in the official documentation](https://github.com/Lightning-AI/lightning/discussions/9144), zero-stage3 and activation partitioning are used together; however, this pairing holds no particular significance.

Therefore, it is thought that there should be clearer statements regarding the use conditions for activation partitioning, beyond simply specifying that it should be used with zero3 + mp.

cc @borda @awaelchli

Start with the DeepSpeedStrategy API documentation entry for partition_activations, then compare its wording with the linked DeepSpeed activation-checkpointing and pipeline-parallelism documentation. Done means the documented conditions and relationship to ZeRO stages, model parallelism, and the mpu object are unambiguous.

Written by the indexing model from the issue text.