[BUG] OPT-66b fails to load from Hugging Face Hub due to layers being spread out across multiple files

Describe the bug
OPT-66b fails to load with DeepSpeed Inference with the following exception:

File "ds_inference_test.py", line 182, in init
    pipe.model = deepspeed.init_inference(pipe.model,
  File "/lib/python3.8/site-packages/deepspeed/__init__.py", line 311, in init_inference
    engine = InferenceEngine(model, config=ds_inference_config)
  File "/lib/python3.8/site-packages/deepspeed/inference/engine.py", line 136, in __init__
    self._apply_injection_policy(config)
  File "/lib/python3.8/site-packages/deepspeed/inference/engine.py", line 363, in _apply_injection_policy
    replace_transformer_layer(client_module,
  File "/lib/python3.8/site-packages/deepspeed/module_inject/replace_module.py", line 563, in replace_transformer_layer
    load_model_with_checkpoint(replaced_module,
  File "/lib/python3.8/site-packages/deepspeed/module_inject/load_checkpoint.py", line 287, in load_model_with_checkpoint
    load_module_recursive(r_module)
  File "/lib/python3.8/site-packages/deepspeed/module_inject/load_checkpoint.py", line 281, in load_module_recursive
    load_module_recursive(
  File "/lib/python3.8/site-packages/deepspeed/module_inject/load_checkpoint.py", line 281, in load_module_recursive
    load_module_recursive(
  File "/lib/python3.8/site-packages/deepspeed/module_inject/load_checkpoint.py", line 281, in load_module_recursive
    load_module_recursive(
  File "/lib/python3.8/site-packages/deepspeed/module_inject/load_checkpoint.py", line 279, in load_module_recursive
    layer_policies[child.__class__](child, prefix + name + '.')
  File "/lib/python3.8/site-packages/deepspeed/module_inject/load_checkpoint.py", line 212, in load_transformer_layer
    container.load_params(module, sd[0], weight_quantizer, mp_replace, prefix)
  File "/lib/python3.8/site-packages/deepspeed/module_inject/containers/opt.py", line 48, in load_params
    maybe_copy_qkv(module.attention,
  File "/lib/python3.8/site-packages/deepspeed/module_inject/policy.py", line 190, in maybe_copy_qkv
    k = sd[src_names[1]]
KeyError: 'model.decoder.layers.9.self_attn.k_proj.weight'

I imagine this is because model.decoder.layers.9.... are spread across multiple files, which is not the case with other OPT models, eg. OPT-30b.

To Reproduce
This can be reproduced using https://github.com/microsoft/DeepSpeedExamples/blob/master/inference/huggingface/text-generation/inference-test.py script - simply do deepspeed --num_gpus 4 inference-test.py --name facebook/opt-66b --batch_size 1 --ds_inference --use_kernel --use_meta_tensor --save_mp_checkpoint_path '/nvme/mp_checkpoint'

Expected behavior
No exception is thrown and the model can be used.

ds_report output

--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
async_io ............... [NO] ....... [OKAY]
cpu_adagrad ............ [NO] ....... [OKAY]
cpu_adam ............... [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
random_ltd ............. [NO] ....... [OKAY]
 [WARNING]  sparse_attn requires a torch version >= 1.5 but detected 2.0
 [WARNING]  using untested triton version (2.0.0), only 1.0.0 is known to be compatible
sparse_attn ............ [NO] ....... [NO]
spatial_inference ...... [NO] ....... [OKAY]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
utils .................. [NO] ....... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/home/ray/anaconda3/lib/python3.8/site-packages/torch']
torch version .................... 2.0.0+cu117
deepspeed install path ........... ['/mnt/cluster_storage/pypi/lib/python3.8/site-packages/deepspeed']
deepspeed info ................... 0.8.3, unknown, unknown
torch cuda version ............... 11.7
torch hip version ................ None
nvcc version ..................... 11.6
deepspeed wheel compiled w. ...... torch 2.0, cuda 11.7

Screenshots
If applicable, add screenshots to help explain your problem.

System info (please complete the following information):

• OS: Ubuntu 20.04.6 LTS
• GPU count and types: 4 x g4dn.12xlarge nodes (4 x 4 T4)
• Transformers 4.27.2
• Python 3.8.13

Start with deepspeed/module_inject/load_checkpoint.py, containers/opt.py, and policy.py, following the failing load_transformer_layer path from the traceback. Use the provided inference-test.py reproduction with facebook/opt-66b and inspect the indexed checkpoint layout. Done means OPT-66b loads through DeepSpeed Inference without the KeyError.