FSDP: add a way to sync params from a given rank

For DDP, rank 0's weight is synced to all ranks before forward. For FSDP, it would be nice to have a way to do this so that different weights from different ranks will be consistent before the forward. This can be easier for model init since we can just init a single rank and sync to all ranks.

We need to do this without GPU OOMs.

cc: @prigoyal @myleott @sshleifer

Start by comparing the existing DDP rank-0 parameter synchronization with FSDP's parameter and forward paths. Design a way to synchronize parameters from a selected rank before forward while avoiding GPU OOMs, then add coverage for consistent weights across ranks and the memory constraint.

Written by the indexing model from the issue text.