[REQUEST] cpu offload needs a max cpu memory config + pointers to cgroups/cpu oom handlers

Currently, unless a user uses some sort of cgroups or oom prevention monitor when using cpu offload with a huge model the desktop is likely to crash.

I propose that cpu and nvme offload define a new config where a user can tell how much cpu memory they can dedicate to the offload and if deepspeed sees it is not enough it should assert rather than continue gobbling up all available memory and bringing the host down to its knees.

and of course if a user didn't define one it should be easy to dynamically look at the available memory and tell the user if it's too low to proceed. This of course doesn't prevent a crash if some other program decides to consume more cpu memory moments later and which may still lead to a situation where deepspeed will consume all of the available memory. But this is a low hanging fruit towards "ease of use".

And perhaps documenting some pointers at how to use cgroups (which in v2 of cgroups is much more complicated to setup). e.g. I currently use with 128GB of cpu RAM:

systemd-run --user --scope -p MemoryHigh=100G -p MemoryMax=110G -p MemorySwapMax=64G bash
deepspeed ...

that's if I'm not running large memory programs.

At 100GB cgroups start throttling the program and at 110GB it will kill it.

Start by locating the CPU and NVMe offload configuration and any existing memory checks. Review the cgroups v2 and systemd-run example in the issue; completion would require an agreed configuration and behavior for insufficient CPU memory, plus documented cgroups guidance.