[request] more intutive api for GatheredParameters read_only vs. write

Currently one has to remember to add modifier_rank=0 to modify params.

deepspeed.zero.GatheredParameters(params, modifier_rank=0)

this is very unintuitive API IMHO and it won't fail if you forget to add it. Often that leads to subtle errors.

I already started using modifier_rank=None for read-only when I don't need to set it - just to remind me that it's a crucial arg to think about, when copy-n-pasting some code.

Could we change this API to something more human meaningful and more difficult to forget to set?

Here are a few possible ideas:

1. add a required named flag, e.g. modify, so now we will have:

# just going to use the data 
deepspeed.zero.GatheredParameters(params, modify=False, modifier_rank=None)
# going to use and modify data
deepspeed.zero.GatheredParameters(params, modify=True, modifier_rank=0)

and if modify==True it will check that modifier_rank is enforced? Actually it can set it automatically

• modify=True => modifier_rank=0 (unless it was set explicitly)
• modify=False => modifier_rank=None ( asserts if non-None was passed

since that's the case most of the time.

So the API becomes a very simple and easy to use:

# just going to use the data 
deepspeed.zero.GatheredParameters(params, modify=False)
# going to use and modify data
deepspeed.zero.GatheredParameters(params, modify=True)

and of course one can override modifier_rank for the modifying case.

I'm totally open to perhaps even more intuitive names.

2. Perhaps read_only is a better one?

# just going to use the data 
deepspeed.zero.GatheredParameters(params, read_only=True)
# going to use and modify data
deepspeed.zero.GatheredParameters(params, read_only=False)

It won't break any back-compat API if we make it optional, but will be helpful to always use in the code. But of course best to make it required - though it'd not be back-compat.

3. Yet another idea, adding 2 wrappers:

# just going to use the data 
deepspeed.zero.GatheredParametersRead(params)
# going to use and modify data
deepspeed.zero.GatheredParametersWrite(params)

Even simpler and more straightforward.

I like this last idea the most as it's the most readable, and one can still use the current API if they want to.

@samyam, @jeffra, @ShadenSmith

Start by locating the GatheredParameters entry point and reviewing how modifier_rank distinguishes read-only from modifying use. Compare the proposed modify, read_only, and wrapper APIs against existing callers and backward-compatibility needs. Done means an agreed API direction with clear behavior for defaults and modifier_rank overrides.