Address FSDP + manual optimization

Bug description

In manual optimization, the user can call self.backward() anywhere in training_step(). There are no limitations for this in single-device execution, but for distributed strategies there are challenges associated with that.

In DDP, we solve that problem by disabling a backward hook before calling the actual backward:
https://github.com/Lightning-AI/pytorch-lightning/blob/6cfc590716cbf52e09033ae11ebee10864ef7589/src/lightning/pytorch/strategies/ddp.py#L316-L317

However, such a mechanism doesn't exist for FSDP, and calling backward during "forward" is not supported in sharded models. We should investigate whether it is ok to do this from the root fsdp model or not, and discuss possible workarounds if there are issues.

What version are you seeing the problem on?

master

How to reproduce the bug

No response

Error messages and logs

# Error messages and logs here please

Environment

#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0):
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
#- Running environment of LightningApp (e.g. local, cloud):

More info

No response

cc @awaelchli @carmocca

Start with the manual-optimization path around training_step() and compare the backward-hook handling in src/lightning/pytorch/strategies/ddp.py at the linked lines. Investigate whether invoking backward from the root FSDP model is supported, and identify viable workarounds for sharded models. The issue has no reproduction or test location, so done is not precisely defined.