Size mismatch for base_model.model.transformer for bloomz model finetuning

After finetuning the bigscience/bloomz-7b1, I encountered this issue while doing evaluation.

│ envs/lmflow/lib/python3.9/site-packages/peft/utils/save_and_load.py:74 │
│ in set_peft_model_state_dict │
│ │
│ 71 │ │ peft_model_state_dict (dict): The state dict of the Peft model. │
│ 72 │ """ │
│ 73 │ │
│ ❱ 74 │ model.load_state_dict(peft_model_state_dict, strict=False) │
│ 75 │ if model.peft_config.peft_type != PeftType.LORA: │
│ 76 │ │ model.prompt_encoder.embedding.load_state_dict( │
│ 77 │ │ │ {"weight": peft_model_state_dict["prompt_embeddings"]}, strict=True │
│ │
│ envs/lmflow/lib/python3.9/site-packages/torch/nn/modules/module.py:160 │
│ 4 in load_state_dict │
│ │
│ 1601 │ │ │ │ │ │ ', '.join('"{}"'.format(k) for k in missing_keys))) │
│ 1602 │ │ │
│ 1603 │ │ if len(error_msgs) > 0: │
│ ❱ 1604 │ │ │ raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( │
│ 1605 │ │ │ │ │ │ │ self.class.name, "\n\t".join(error_msgs))) │
│ 1606 │ │ return _IncompatibleKeys(missing_keys, unexpected_keys) │
│ 1607 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Error(s) in loading state_dict for PeftModelForCausalLM:
size mismatch for base_model.model.transformer.h.0.self_attention.query_key_value.lora_A.weight: copying a param with shape torch.Size([0]) from
checkpoint, the shape in current model is torch.Size([16, 4096]).
size mismatch for base_model.model.transformer.h.0.self_attention.query_key_value.lora_B.weight: copying a param with shape torch.Size([0]) from
checkpoint, the shape in current model is torch.Size([8192, 8, 1]).
size mismatch for base_model.model.transformer.h.1.self_attention.query_key_value.lora_A.weight: copying a param with shape torch.Size([0]) from
checkpoint, the shape in current model is torch.Size([16, 4096]).

Start with the bloomz-7b1 fine-tuning and evaluation flow, then inspect peft/utils/save_and_load.py at set_peft_model_state_dict and the PyTorch load_state_dict call shown in the traceback. Reproduce the checkpoint-loading failure and determine why the saved LoRA tensors have shape [0] instead of matching the current model; done means evaluation loads the fine-tuned checkpoint without the reported size mismatch.