[notebook] running deepspeed w/o a launcher

There is no laucher inside a notebook (jupyter / colab), so a user needs to work around it with:

# 1. mock up the launcher
import os
os.environ['MASTER_ADDR'] = 'localhost'
os.environ['MASTER_PORT'] = '9994'
os.environ['RANK'] = "0"
os.environ['LOCAL_RANK'] = "0"
os.environ['WORLD_SIZE'] = "1"

# 2. fix local_rank arg if it wasn't passed, in HF Trainer case it is:
if args.local_rank == -1: 
    args.local_rank = 0

# 3. finally init deepspeed dist and set the default device
deepspeed.init_distributed()
device = torch.device("cuda", self.local_rank)

Request:

1. It'd be useful if DeepSpeed provided a wrapper for the (1) first group of env setting commands with ability to override any of these values, but not needing to pass any of those either - i.e. good defaults. Not sure - perhaps deepspeed.single_process_no_laucher_dist_setup?

I wonder if this should be made as part of deepspeed.init_distributed() - but for the framework it'd be difficult to know how it was called.

2. making sure to check that the port is free for binding when picking a port on behalf of the user

and when this is done it'd be good to document somewhere how to run deepspeed in a notebook. probably where launcher is discussed.

The other solution is to have mpi4py a requirement - I'm not sure if it's an easy one or not (potential problems with install?). If that package is installed then there is no need for any of the code from group 1 as it automatically figures it all out. But if we go this way, it shouldn't be an optional dependency I think. I think that package depends on other packages too.

Perhaps this is a simpler solution as it requires no API changes. I actually prefer this one, but I don't know whether mpi4py is an easy install everywhere...

Thank you!

Start by reading deepspeed.init_distributed() and the launcher behavior described in the issue, then reproduce the current environment setup in a Jupyter or Colab notebook. Compare the proposed wrapper and mpi4py approaches, including free-port handling and value overrides. Done requires an agreed implementation path plus notebook documentation showing the supported setup.