NVIDIA / NVIDIA/Megatron-LM

Megatron Multimodal: model_provider() got an unexpected keyword argument 'config'

Open
#3,269 0 comments 0 reactions 1 assignee Claimed by @yaoyu-33 View on GitHub
bug
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 6h
Merged PRs (30d)
271

Description

**Describe the bug**

I am following the guide: https://github.com/NVIDIA/Megatron-LM/tree/main/examples/multimodal

Step 5: Run the following script to pretrain a llava model for image captioning:

cd
examples/multimodal/pretrain_mistral_clip.sh

I got

```
model_provider() got an unexpected keyword argument 'config'

in the training.py file
```

Full error
```
[after in-process setup and before initialize_megatron] datetime: 2026-02-05 10:11:54.935362
[after in-job setup and before initialize_megatron] datetime: 2026-02-05 10:11:54.973006
[after megatron is initialized] datetime: 2026-02-05 10:12:08.067141
Failed to inspect checkpoint in ./output/mcore-llava-mistral-7b-instruct-clip336-pretraining/checkpoints: _load_base_checkpoint() missing 1 required positional argument: 'args'

[rank1]: Traceback (most recent call last):
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/examples/multimodal/train.py", line 384, in
[rank1]: pretrain(
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/megatron/training/training.py", line 902, in pretrain
[rank1]: model, optimizer, opt_param_scheduler = setup_model_and_optimizer(
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/megatron/training/training.py", line 1455, in setup_model_and_optimizer
[rank1]: model = get_model(model_provider_func, model_type, wrap_with_ddp=wrap_with_ddp)
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/megatron/training/training.py", line 1228, in get_model
[rank1]: model = build_model()
[rank1]: ^^^^^^^^^^^^^
[rank1]: File "/lustre/fsw/coreai_prod_infbench/faradawny/Megatron-LM/megatron/training/training.py", line 1214, in build_model
[rank1]: model = model_provider_func(
[rank1]: ^^^^^^^^^^^^^^^^^^^^
[rank1]: TypeError: model_provider() got an unexpected keyword argument 'config'
```

**Steps/Code to reproduce bug**

Please list *minimal* steps or code snippet for us to be able to reproduce the bug.

A helpful guide on on how to craft a minimal bug report http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports.

**Expected behavior**

A clear and concise description of what you expected to happen.

**Additional context**

Add any other context about the problem here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.