[doc] configuring `offload_*` param sections

https://github.com/microsoft/DeepSpeed/issues/998 tackles the aio param section, but we still have no user guide for the new "offload_optimizer" and "offload_param" sections. We have:

            "offload_optimizer": {​​​​​
                "device": "nvme",
                "nvme_path": "/local_nvme",
                "pin_memory": true,
                "buffer_count": 4,
                "fast_init": false
            }​​​​​,
            "offload_param": {​​​​​
                "device": "nvme",
                "nvme_path": "/local_nvme",
                "pin_memory": true,
                "buffer_count": 5,
                "buffer_size": 1e8,
                "max_in_cpu": 1e9
            }​​​​​

other than device, nvme_path and pin_memory which are pretty obvious, the rest have super-terse descriptions and a user will have no idea how to configure those. Let's write a guide to how these values should be chosen.

I copied the descriptions and defaults that already exist and tried to ask the right questions, so if you could answer those I think that would be a great start.

Thank you!

Optimizer

• buffer_count: default 4: Number of buffers in buffer pool for optimizer state offloading to NVMe. This should be at least the number of states maintained per parameter by the optimizer. For example, Adam optimizer has 4 states (parameter, gradient, momentum, and variance)

Q: why "at least" - is it more efficient to have it bigger?
Q: what's the impact on memory footprint (CPU/NVMe)

• fast_init: default false. Enable fast optimizer initialization when offloading to NVMe.

Q: why is it false by default?

Param

• buffer_count: default 5: Number of buffers in buffer pool for parameter offloading to NVMe.

Q: why 5, what are the correlations to other params?

• buffer_size: default 1e8: Size of buffers in buffer pool for parameter offloading to NVMe.

Q: how do we get to this number and how it correlates with other config params?
Q: what's the impact on memory footprint (CPU/NVMe)

• max_in_cpu: default 1e9: Number of parameter elements to maintain in CPU memory when offloading to NVMe is enabled

Q: how do we get to this number and how it correlates with other config params?
Q: what's the impact on memory footprint (CPU/NVMe)

Start by reviewing issue #998 on the aio parameter section and the existing descriptions and defaults for offload_optimizer and offload_param. Research and document how buffer_count, fast_init, buffer_size, and max_in_cpu should be chosen, including their memory and performance effects. Done means users have a practical guide for configuring both sections.