[BUG] `DeepSpeedCPUAdam` compatibility on ROCm

Describe the bug
Hi,

import torch
from deepspeed.ops.adam import DeepSpeedCPUAdam

fused_adam = DeepSpeedCPUAdam([torch.rand(10)])

yields [WARNING] cpu_adam cuda is missing or is incompatible with installed torch, only cpu ops can be compiled! on a ROCm install of PyTorch & DeepSpeed. Is this expected?

Thank you!

To Reproduce
Use torch==2.1.1+rocm5.6 & deepspeed==0.12.4.

Expected behavior
A clear and concise description of what you expected to happen.

ds_report output

--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
async_io ............... [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
cpu_adam ............... [NO] ....... [OKAY]
cpu_adagrad ............ [NO] ....... [OKAY]
cpu_lion ............... [NO] ....... [OKAY]
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
evoformer_attn ......... [NO] ....... [NO]
fused_lamb ............. [NO] ....... [OKAY]
fused_lion ............. [NO] ....... [OKAY]
inference_core_ops ..... [NO] ....... [OKAY]
cutlass_ops ............ [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
ragged_device_ops ...... [NO] ....... [OKAY]
ragged_ops ............. [NO] ....... [OKAY]
random_ltd ............. [NO] ....... [OKAY]
 [WARNING]  sparse_attn is not compatible with ROCM
sparse_attn ............ [NO] ....... [NO]
spatial_inference ...... [NO] ....... [OKAY]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/usr/local/lib/python3.10/dist-packages/torch']
torch version .................... 2.1.1+rocm5.6
deepspeed install path ........... ['/usr/local/lib/python3.10/dist-packages/deepspeed']
deepspeed info ................... 0.12.4, unknown, unknown
torch cuda version ............... None
torch hip version ................ 5.6.31061-8c743ae5d
nvcc version ..................... None
deepspeed wheel compiled w. ...... torch 2.1, hip 5.6
shared memory (/dev/shm) size .... 64.00 GB

System info (please complete the following information):

• GPU count and types: MI250

Start from the DeepSpeedCPUAdam import and instantiation shown in the issue, then inspect the ds_report output for the cpu_adam compatibility warning on torch==2.1.1+rocm5.6 and deepspeed==0.12.4. Determine whether the warning is expected on the MI250/ROCm environment and document or verify the intended behavior; the issue does not name a file or test.