Megatron Multimodal: model_provider() got an unexpected keyword argument 'config'
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 271
Description
**Describe the bug**
I am following the guide: https://github.com/NVIDIA/Megatron-LM/tree/main/examples/multimodal
Step 5: Run the following script to pretrain a llava model for image captioning:
cd
examples/multimodal/pretrain_mistral_clip.sh
I got
```
model_provider() got an unexpected keyword argument 'config'
in the training.py file
```
Full error
```
[after in-process setup and before initialize_megatron] datetime: 2026-02-05 10:11:54.935362
[after in-job setup and before initialize_megatron] datetime: 2026-02-05 10:11:54.973006
[after megatron is initialized] datetime: 2026-02-05 10:12:08.067141
Failed to inspect checkpoint in ./output/mcore-llava-mistral-7b-instruct-clip336-pretraining/checkpoints: _load_base_checkpoint() missing 1 required positional argument: 'args'
[rank1]: Traceback (most recent call last):
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/examples/multimodal/train.py", line 384, in
[rank1]: pretrain(
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/megatron/training/training.py", line 902, in pretrain
[rank1]: model, optimizer, opt_param_scheduler = setup_model_and_optimizer(
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/megatron/training/training.py", line 1455, in setup_model_and_optimizer
[rank1]: model = get_model(model_provider_func, model_type, wrap_with_ddp=wrap_with_ddp)
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/megatron/training/training.py", line 1228, in get_model
[rank1]: model = build_model()
[rank1]: ^^^^^^^^^^^^^
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/megatron/training/training.py", line 1214, in build_model
[rank1]: model = model_provider_func(
[rank1]: ^^^^^^^^^^^^^^^^^^^^
[rank1]: TypeError: model_provider() got an unexpected keyword argument 'config'
```
**Steps/Code to reproduce bug**
Please list *minimal* steps or code snippet for us to be able to reproduce the bug.
A helpful guide on on how to craft a minimal bug report http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports.
**Expected behavior**
A clear and concise description of what you expected to happen.
**Additional context**
Add any other context about the problem here.
Contributor guide
Assessment
This issue has not been assessed yet.