does deepspeed support pure bf16 training?

training 70B cost too large gpu memory
when training 70+B model, the cost of gpu memory is too large, as mentioned in the deepspeed deepspeed-readthedocs-io-en-stable.pdf, the total memory was about 18xN, N is the number of params, if offload, the cpu memory may need 2T+. the main factor of this is the 32bit of (adam optmizer+gradient+copy of model param)

expect deepspeed can support pure bf16 training
if using pure bf16, which means all param(model+gradient+optimizer) only using bf16, the cost of memory may down to 8xN, and in most cases, bf16 is enough, I have test this using llama-recipe(https://github.com/meta-llama/llama-recipes). the result of pure bf16 traing is very similar to deepspeed mixed precision, but llama-recipe using pure bf16 can train 70B in one node(8x80G A800) by freeze half layers

using mmap technology to offload optmizer states to disk
is it possible to offload gradient and optimizer to disk, and then using mmap load it quickly?

Additional context
Add any other context or screenshots about the feature request here.

Start by reading the linked DeepSpeed documentation PDF and comparing the pure-bf16 behavior described through llama-recipes. Identify the existing training and optimizer-offload entry points, then define completion as validated pure-bf16 support and a measured answer on disk-backed gradient or optimizer offloading.