[BUG] Unable to load model from TimmAutoModelForImagePrediction

**Bug Report Checklist**

- [X] I provided code that demonstrates a minimal reproducible example.
- [X] I confirmed bug exists on the latest stable version of AutoGluon.
- [x] I confirmed bug exists on the latest mainline of AutoGluon via source install.

**Describe the bug**

I trained a img classify model with the code below.
After finished training, I tried to load the model in jupyter notebook and I got this error:

> Load pretrained checkpoint: /root/cs/best_q_mixup_aug_1/model.ckpt
>
> ---------------------------------------------------------------------------
> RuntimeError Traceback (most recent call last)
> Cell In[32], line 2
> 1 from autogluon.multimodal import MultiModalPredictor
> ----> 2 predictor = MultiModalPredictor.load("/root/cs/best_q_mixup_aug_1")
>
> File ~/miniconda3/lib/python3.10/site-packages/autogluon/multimodal/predictor.py:817, in MultiModalPredictor.load(cls, path, resume, verbosity)
> 814 else:
> 815 learner_class = BaseLearner
> --> 817 predictor._learner = learner_class.load(path=path, resume=resume, verbosity=verbosity)
> 818 return predictor
>
> File ~/miniconda3/lib/python3.10/site-packages/autogluon/multimodal/learners/base.py:2368, in BaseLearner.load(cls, path, resume, verbosity)
> 2359 learner._data_processors = create_fusion_data_processors(
> 2360 config=learner._config,
> 2361 model=learner._model,
> 2362 )
> 2363 load_path, ckpt_path = get_load_ckpt_paths(
> 2364 ckpt_path=ckpt_path,
> 2365 dir_path=dir_path,
> 2366 resume=resume,
> 2367 )
> -> 2368 learner._load_state_dict(
> 2369 path=load_path,
> 2370 strict=not peft,
> 2371 )
> 2372 learner._ckpt_path = ckpt_path
> 2373 loss_func = get_loss_func(
> 2374 problem_type=learner._problem_type,
> 2375 mixup_active=False,
> (...)
> 2378 num_classes=learner._output_shape,
> 2379 )
>
> File ~/miniconda3/lib/python3.10/site-packages/autogluon/multimodal/learners/base.py:2120, in BaseLearner._load_state_dict(self, state_dict, path, prefix, strict)
> 2117 buffer_names_to_filter = [k for k in buffer_names if k not in self._model.state_dict().keys()]
> 2118 state_dict = {k: v for k, v in state_dict.items() if k not in buffer_names_to_filter}
> -> 2120 load_result = self._model.load_state_dict(state_dict, strict=strict)
> 2121 assert (
> 2122 len(load_result.unexpected_keys) == 0
> 2123 ), f"Load model failed, unexpected keys {load_result.unexpected_keys.__str__()}"
>
> File ~/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py:2189, in Module.load_state_dict(self, state_dict, strict, assign)
> 2184 error_msgs.insert(
> 2185 0, 'Missing key(s) in state_dict: {}. '.format(
> 2186 ', '.join(f'"{k}"' for k in missing_keys)))
> 2188 if len(error_msgs) > 0:
> -> 2189 raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
> 2190 self.__class__.__name__, "\n\t".join(error_msgs)))
> 2191 return _IncompatibleKeys(missing_keys, unexpected_keys)
>
> RuntimeError: Error(s) in loading state_dict for TimmAutoModelForImagePrediction:
> Missing key(s) in state_dict: "model.patch_embed.proj.weight", "model.patch_embed.proj.bias", "model.patch_embed.norm.weight", "model.patch_embed.norm.bias", "model.layers.0.blocks.0.norm1.weight", "model.layers.0.blocks.0.norm1.bias", "model.layers.0.blocks.0.attn.relative_position_bias_table", "model.layers.0.blocks.0.attn.qkv.weight", "model.layers.0.blocks.0.attn.qkv.bias", "model.layers.0.blocks.0.attn.proj.weight", "model.layers.0.blocks.0.attn.proj.bias", "model.layers.0.blocks.0.norm2.weight", "model.layers.0.blocks.0.norm2.bias", "model.layers.0.blocks.0.mlp.fc1.weight", "model.layers.0.blocks.0.mlp.fc1.bias", "model.layers.0.blocks.0.mlp.fc2.weight", "model.layers.0.blocks.0.mlp.fc2.bias", "model.layers.0.blocks.1.norm1.weight", "model.layers.0.blocks.1.norm1.bias", "model.layers.0.blocks.1.attn.relative_position_bias_table", "model.layers.0.blocks.1.attn.qkv.weight", "model.layers.0.blocks.1.attn.qkv.bias", "model.layers.0.blocks.1.attn.proj.weight", "model.layers.0.blocks.1.attn.proj.bias", "model.layers.0.blocks.1.norm2.weight", "model.layers.0.blocks.1.norm2.bias", "model.layers.0.blocks.1.mlp.fc1.weight", "model.layers.0.blocks.1.mlp.fc1.bias", "model.layers.0.blocks.1.mlp.fc2.weight", "model.layers.0.blocks.1.mlp.fc2.bias", "model.layers.1.downsample.norm.weight", "model.layers.1.downsample.norm.bias", "model.layers.1.downsample.reduction.weight", "model.layers.1.blocks.0.norm1.weight", "model.layers.1.blocks.0.norm1.bias", "model.layers.1.blocks.0.attn.relative_position_bias_table", "model.layers.1.blocks.0.attn.qkv.weight", "model.layers.1.blocks.0.attn.qkv.bias", "model.layers.1.blocks.0.attn.proj.weight", "model.layers.1.blocks.0.attn.proj.bias", "model.layers.1.blocks.0.norm2.weight", "model.layers.1.blocks.0.norm2.bias", "model.layers.1.blocks.0.mlp.fc1.weight", "model.layers.1.blocks.0.mlp.fc1.bias", "model.layers.1.blocks.0.mlp.fc2.weight", "model.layers.1.blocks.0.mlp.fc2.bias", "model.layers.1.blocks.1.norm1.weight", "model.layers.1.blocks.1.norm1.bias", "model.layers.1.blocks.1.attn.relative_position_bias_table", "model.layers.1.blocks.1.attn.qkv.weight", "model.layers.1.blocks.1.attn.qkv.bias", "model.layers.1.blocks.1.attn.proj.weight", "model.layers.1.blocks.1.attn.proj.bias", "model.layers.1.blocks.1.norm2.weight", "model.layers.1.blocks.1.norm2.bias", "model.layers.1.blocks.1.mlp.fc1.weight", "model.layers.1.blocks.1.mlp.fc1.bias", "model.layers.1.blocks.1.mlp.fc2.weight", "model.layers.1.blocks.1.mlp.fc2.bias", "model.layers.2.downsample.norm.weight", "model.layers.2.downsample.norm.bias", "model.layers.2.downsample.reduction.weight", "model.layers.2.blocks.0.norm1.weight", "model.layers.2.blocks.0.norm1.bias", "model.layers.2.blocks.0.attn.relative_position_bias_table", "model.layers.2.blocks.0.attn.qkv.weight", "model.layers.2.blocks.0.attn.qkv.bias", "model.layers.2.blocks.0.attn.proj.weight", "model.layers.2.blocks.0.attn.proj.bias", "model.layers.2.blocks.0.norm2.weight", "model.layers.2.blocks.0.norm2.bias", "model.layers.2.blocks.0.mlp.fc1.weight", "model.layers.2.blocks.0.mlp.fc1.bias", "model.layers.2.blocks.0.mlp.fc2.weight", "model.layers.2.blocks.0.mlp.fc2.bias", "model.layers.2.blocks.1.norm1.weight", "model.layers.2.blocks.1.norm1.bias", "model.layers.2.blocks.1.attn.relative_position_bias_table", "model.layers.2.blocks.1.attn.qkv.weight", "model.layers.2.blocks.1.attn.qkv.bias", "model.layers.2.blocks.1.attn.proj.weight", "model.layers.2.blocks.1.attn.proj.bias", "model.layers.2.blocks.1.norm2.weight", "model.layers.2.blocks.1.norm2.bias", "model.layers.2.blocks.1.mlp.fc1.weight", "model.layers.2.blocks.1.mlp.fc1.bias", "model.layers.2.blocks.1.mlp.fc2.weight", "model.layers.2.blocks.1.mlp.fc2.bias", "model.layers.2.blocks.2.norm1.weight", "model.layers.2.blocks.2.norm1.bias", "model.layers.2.blocks.2.attn.relative_position_bias_table", "model.layers.2.blocks.2.attn.qkv.weight", "model.layers.2.blocks.2.attn.qkv.bias", "model.layers.2.blocks.2.attn.proj.weight", "model.layers.2.blocks.2.attn.proj.bias", "model.layers.2.blocks.2.norm2.weight", "model.layers.2.blocks.2.norm2.bias", "model.layers.2.blocks.2.mlp.fc1.weight", "model.layers.2.blocks.2.mlp.fc1.bias", "model.layers.2.blocks.2.mlp.fc2.weight", "model.layers.2.blocks.2.mlp.fc2.bias", "model.layers.2.blocks.3.norm1.weight", "model.layers.2.blocks.3.norm1.bias", "model.layers.2.blocks.3.attn.relative_position_bias_table", "model.layers.2.blocks.3.attn.qkv.weight", "model.layers.2.blocks.3.attn.qkv.bias", "model.layers.2.blocks.3.attn.proj.weight", "model.layers.2.blocks.3.attn.proj.bias", "model.layers.2.blocks.3.norm2.weight", "model.layers.2.blocks.3.norm2.bias", "model.layers.2.blocks.3.mlp.fc1.weight", "model.layers.2.blocks.3.mlp.fc1.bias", "model.layers.2.blocks.3.mlp.fc2.weight", "model.layers.2.blocks.3.mlp.fc2.bias", "model.layers.2.blocks.4.norm1.weight", "model.layers.2.blocks.4.norm1.bias", "model.layers.2.blocks.4.attn.relative_position_bias_table", "model.layers.2.blocks.4.attn.qkv.weight", "model.layers.2.blocks.4.attn.qkv.bias", "model.layers.2.blocks.4.attn.proj.weight", "model.layers.2.blocks.4.attn.proj.bias", "model.layers.2.blocks.4.norm2.weight", "model.layers.2.blocks.4.norm2.bias", "model.layers.2.blocks.4.mlp.fc1.weight", "model.layers.2.blocks.4.mlp.fc1.bias", "model.layers.2.blocks.4.mlp.fc2.weight", "model.layers.2.blocks.4.mlp.fc2.bias", "model.layers.2.blocks.5.norm1.weight", "model.layers.2.blocks.5.norm1.bias", "model.layers.2.blocks.5.attn.relative_position_bias_table", "model.layers.2.blocks.5.attn.qkv.weight", "model.layers.2.blocks.5.attn.qkv.bias", "model.layers.2.blocks.5.attn.proj.weight", "model.layers.2.blocks.5.attn.proj.bias", "model.layers.2.blocks.5.norm2.weight", "model.layers.2.blocks.5.norm2.bias", "model.layers.2.blocks.5.mlp.fc1.weight", "model.layers.2.blocks.5.mlp.fc1.bias", "model.layers.2.blocks.5.mlp.fc2.weight", "model.layers.2.blocks.5.mlp.fc2.bias", "model.layers.2.blocks.6.norm1.weight", "model.layers.2.blocks.6.norm1.bias", "model.layers.2.blocks.6.attn.relative_position_bias_table", "model.layers.2.blocks.6.attn.qkv.weight", "model.layers.2.blocks.6.attn.qkv.bias", "model.layers.2.blocks.6.attn.proj.weight", "model.layers.2.blocks.6.attn.proj.bias", "model.layers.2.blocks.6.norm2.weight", "model.layers.2.blocks.6.norm2.bias", "model.layers.2.blocks.6.mlp.fc1.weight", "model.layers.2.blocks.6.mlp.fc1.bias", "model.layers.2.blocks.6.mlp.fc2.weight", "model.layers.2.blocks.6.mlp.fc2.bias", "model.layers.2.blocks.7.norm1.weight", "model.layers.2.blocks.7.norm1.bias", "model.layers.2.blocks.7.attn.relative_position_bias_table", "model.layers.2.blocks.7.attn.qkv.weight", "model.layers.2.blocks.7.attn.qkv.bias", "model.layers.2.blocks.7.attn.proj.weight", "model.layers.2.blocks.7.attn.proj.bias", "model.layers.2.blocks.7.norm2.weight", "model.layers.2.blocks.7.norm2.bias", "model.layers.2.blocks.7.mlp.fc1.weight", "model.layers.2.blocks.7.mlp.fc1.bias", "model.layers.2.blocks.7.mlp.fc2.weight", "model.layers.2.blocks.7.mlp.fc2.bias", "model.layers.2.blocks.8.norm1.weight", "model.layers.2.blocks.8.norm1.bias", "model.layers.2.blocks.8.attn.relative_position_bias_table", "model.layers.2.blocks.8.attn.qkv.weight", "model.layers.2.blocks.8.attn.qkv.bias", "model.layers.2.blocks.8.attn.proj.weight", "model.layers.2.blocks.8.attn.proj.bias", "model.layers.2.blocks.8.norm2.weight", "model.layers.2.blocks.8.norm2.bias", "model.layers.2.blocks.8.mlp.fc1.weight", "model.layers.2.blocks.8.mlp.fc1.bias", "model.layers.2.blocks.8.mlp.fc2.weight", "model.layers.2.blocks.8.mlp.fc2.bias", "model.layers.2.blocks.9.norm1.weight", "model.layers.2.blocks.9.norm1.bias", "model.layers.2.blocks.9.attn.relative_position_bias_table", "model.layers.2.blocks.9.attn.qkv.weight", "model.layers.2.blocks.9.attn.qkv.bias", "model.layers.2.blocks.9.attn.proj.weight", "model.layers.2.blocks.9.attn.proj.bias", "model.layers.2.blocks.9.norm2.weight", "model.layers.2.blocks.9.norm2.bias", "model.layers.2.blocks.9.mlp.fc1.weight", "model.layers.2.blocks.9.mlp.fc1.bias", "model.layers.2.blocks.9.mlp.fc2.weight", "model.layers.2.blocks.9.mlp.fc2.bias", "model.layers.2.blocks.10.norm1.weight", "model.layers.2.blocks.10.norm1.bias", "model.layers.2.blocks.10.attn.relative_position_bias_table", "model.layers.2.blocks.10.attn.qkv.weight", "model.layers.2.blocks.10.attn.qkv.bias", "model.layers.2.blocks.10.attn.proj.weight", "model.layers.2.blocks.10.attn.proj.bias", "model.layers.2.blocks.10.norm2.weight", "model.layers.2.blocks.10.norm2.bias", "model.layers.2.blocks.10.mlp.fc1.weight", "model.layers.2.blocks.10.mlp.fc1.bias", "model.layers.2.blocks.10.mlp.fc2.weight", "model.layers.2.blocks.10.mlp.fc2.bias", "model.layers.2.blocks.11.norm1.weight", "model.layers.2.blocks.11.norm1.bias", "model.layers.2.blocks.11.attn.relative_position_bias_table", "model.layers.2.blocks.11.attn.qkv.weight", "model.layers.2.blocks.11.attn.qkv.bias", "model.layers.2.blocks.11.attn.proj.weight", "model.layers.2.blocks.11.attn.proj.bias", "model.layers.2.blocks.11.norm2.weight", "model.layers.2.blocks.11.norm2.bias", "model.layers.2.blocks.11.mlp.fc1.weight", "model.layers.2.blocks.11.mlp.fc1.bias", "model.layers.2.blocks.11.mlp.fc2.weight", "model.layers.2.blocks.11.mlp.fc2.bias", "model.layers.2.blocks.12.norm1.weight", "model.layers.2.blocks.12.norm1.bias", "model.layers.2.blocks.12.attn.relative_position_bias_table", "model.layers.2.blocks.12.attn.qkv.weight", "model.layers.2.blocks.12.attn.qkv.bias", "model.layers.2.blocks.12.attn.proj.weight", "model.layers.2.blocks.12.attn.proj.bias", "model.layers.2.blocks.12.norm2.weight", "model.layers.2.blocks.12.norm2.bias", "model.layers.2.blocks.12.mlp.fc1.weight", "model.layers.2.blocks.12.mlp.fc1.bias", "model.layers.2.blocks.12.mlp.fc2.weight", "model.layers.2.blocks.12.mlp.fc2.bias", "model.layers.2.blocks.13.norm1.weight", "model.layers.2.blocks.13.norm1.bias", "model.layers.2.blocks.13.attn.relative_position_bias_table", "model.layers.2.blocks.13.attn.qkv.weight", "model.layers.2.blocks.13.attn.qkv.bias", "model.layers.2.blocks.13.attn.proj.weight", "model.layers.2.blocks.13.attn.proj.bias", "model.layers.2.blocks.13.norm2.weight", "model.layers.2.blocks.13.norm2.bias", "model.layers.2.blocks.13.mlp.fc1.weight", "model.layers.2.blocks.13.mlp.fc1.bias", "model.layers.2.blocks.13.mlp.fc2.weight", "model.layers.2.blocks.13.mlp.fc2.bias", "model.layers.2.blocks.14.norm1.weight", "model.layers.2.blocks.14.norm1.bias", "model.layers.2.blocks.14.attn.relative_position_bias_table", "model.layers.2.blocks.14.attn.qkv.weight", "model.layers.2.blocks.14.attn.qkv.bias", "model.layers.2.blocks.14.attn.proj.weight", "model.layers.2.blocks.14.attn.proj.bias", "model.layers.2.blocks.14.norm2.weight", "model.layers.2.blocks.14.norm2.bias", "model.layers.2.blocks.14.mlp.fc1.weight", "model.layers.2.blocks.14.mlp.fc1.bias", "model.layers.2.blocks.14.mlp.fc2.weight", "model.layers.2.blocks.14.mlp.fc2.bias", "model.layers.2.blocks.15.norm1.weight", "model.layers.2.blocks.15.norm1.bias", "model.layers.2.blocks.15.attn.relative_position_bias_table", "model.layers.2.blocks.15.attn.qkv.weight", "model.layers.2.blocks.15.attn.qkv.bias", "model.layers.2.blocks.15.attn.proj.weight", "model.layers.2.blocks.15.attn.proj.bias", "model.layers.2.blocks.15.norm2.weight", "model.layers.2.blocks.15.norm2.bias", "model.layers.2.blocks.15.mlp.fc1.weight", "model.layers.2.blocks.15.mlp.fc1.bias", "model.layers.2.blocks.15.mlp.fc2.weight", "model.layers.2.blocks.15.mlp.fc2.bias", "model.layers.2.blocks.16.norm1.weight", "model.layers.2.blocks.16.norm1.bias", "model.layers.2.blocks.16.attn.relative_position_bias_table", "model.layers.2.blocks.16.attn.qkv.weight", "model.layers.2.blocks.16.attn.qkv.bias", "model.layers.2.blocks.16.attn.proj.weight", "model.layers.2.blocks.16.attn.proj.bias", "model.layers.2.blocks.16.norm2.weight", "model.layers.2.blocks.16.norm2.bias", "model.layers.2.blocks.16.mlp.fc1.weight", "model.layers.2.blocks.16.mlp.fc1.bias", "model.layers.2.blocks.16.mlp.fc2.weight", "model.layers.2.blocks.16.mlp.fc2.bias", "model.layers.2.blocks.17.norm1.weight", "model.layers.2.blocks.17.norm1.bias", "model.layers.2.blocks.17.attn.relative_position_bias_table", "model.layers.2.blocks.17.attn.qkv.weight", "model.layers.2.blocks.17.attn.qkv.bias", "model.layers.2.blocks.17.attn.proj.weight", "model.layers.2.blocks.17.attn.proj.bias", "model.layers.2.blocks.17.norm2.weight", "model.layers.2.blocks.17.norm2.bias", "model.layers.2.blocks.17.mlp.fc1.weight", "model.layers.2.blocks.17.mlp.fc1.bias", "model.layers.2.blocks.17.mlp.fc2.weight", "model.layers.2.blocks.17.mlp.fc2.bias", "model.layers.3.downsample.norm.weight", "model.layers.3.downsample.norm.bias", "model.layers.3.downsample.reduction.weight", "model.layers.3.blocks.0.norm1.weight", "model.layers.3.blocks.0.norm1.bias", "model.layers.3.blocks.0.attn.relative_position_bias_table", "model.layers.3.blocks.0.attn.qkv.weight", "model.layers.3.blocks.0.attn.qkv.bias", "model.layers.3.blocks.0.attn.proj.weight", "model.layers.3.blocks.0.attn.proj.bias", "model.layers.3.blocks.0.norm2.weight", "model.layers.3.blocks.0.norm2.bias", "model.layers.3.blocks.0.mlp.fc1.weight", "model.layers.3.blocks.0.mlp.fc1.bias", "model.layers.3.blocks.0.mlp.fc2.weight", "model.layers.3.blocks.0.mlp.fc2.bias", "model.layers.3.blocks.1.norm1.weight", "model.layers.3.blocks.1.norm1.bias", "model.layers.3.blocks.1.attn.relative_position_bias_table", "model.layers.3.blocks.1.attn.qkv.weight", "model.layers.3.blocks.1.attn.qkv.bias", "model.layers.3.blocks.1.attn.proj.weight", "model.layers.3.blocks.1.attn.proj.bias", "model.layers.3.blocks.1.norm2.weight", "model.layers.3.blocks.1.norm2.bias", "model.layers.3.blocks.1.mlp.fc1.weight", "model.layers.3.blocks.1.mlp.fc1.bias", "model.layers.3.blocks.1.mlp.fc2.weight", "model.layers.3.blocks.1.mlp.fc2.bias", "model.norm.weight", "model.norm.bias", "head.weight", "head.bias".
> Unexpected key(s) in state_dict: "_orig_mod.model.patch_embed.proj.weight", "_orig_mod.model.patch_embed.proj.bias", "_orig_mod.model.patch_embed.norm.weight", "_orig_mod.model.patch_embed.norm.bias", "_orig_mod.model.layers.0.blocks.0.norm1.weight", "_orig_mod.model.layers.0.blocks.0.norm1.bias", "_orig_mod.model.layers.0.blocks.0.attn.relative_position_bias_table", "_orig_mod.model.layers.0.blocks.0.attn.qkv.weight", "_orig_mod.model.layers.0.blocks.0.attn.qkv.bias", "_orig_mod.model.layers.0.blocks.0.attn.proj.weight", "_orig_mod.model.layers.0.blocks.0.attn.proj.bias", "_orig_mod.model.layers.0.blocks.0.norm2.weight", "_orig_mod.model.layers.0.blocks.0.norm2.bias", "_orig_mod.model.layers.0.blocks.0.mlp.fc1.weight", "_orig_mod.model.layers.0.blocks.0.mlp.fc1.bias", "_orig_mod.model.layers.0.blocks.0.mlp.fc2.weight", "_orig_mod.model.layers.0.blocks.0.mlp.fc2.bias", "_orig_mod.model.layers.0.blocks.1.norm1.weight", "_orig_mod.model.layers.0.blocks.1.norm1.bias", "_orig_mod.model.layers.0.blocks.1.attn.relative_position_bias_table", "_orig_mod.model.layers.0.blocks.1.attn.qkv.weight", "_orig_mod.model.layers.0.blocks.1.attn.qkv.bias", "_orig_mod.model.layers.0.blocks.1.attn.proj.weight", "_orig_mod.model.layers.0.blocks.1.attn.proj.bias", "_orig_mod.model.layers.0.blocks.1.norm2.weight", "_orig_mod.model.layers.0.blocks.1.norm2.bias", "_orig_mod.model.layers.0.blocks.1.mlp.fc1.weight", "_orig_mod.model.layers.0.blocks.1.mlp.fc1.bias", "_orig_mod.model.layers.0.blocks.1.mlp.fc2.weight", "_orig_mod.model.layers.0.blocks.1.mlp.fc2.bias", "_orig_mod.model.layers.1.downsample.norm.weight", "_orig_mod.model.layers.1.downsample.norm.bias", "_orig_mod.model.layers.1.downsample.reduction.weight", "_orig_mod.model.layers.1.blocks.0.norm1.weight", "_orig_mod.model.layers.1.blocks.0.norm1.bias", "_orig_mod.model.layers.1.blocks.0.attn.relative_position_bias_table", "_orig_mod.model.layers.1.blocks.0.attn.qkv.weight", "_orig_mod.model.layers.1.blocks.0.attn.qkv.bias", "_orig_mod.model.layers.1.blocks.0.attn.proj.weight", "_orig_mod.model.layers.1.blocks.0.attn.proj.bias", "_orig_mod.model.layers.1.blocks.0.norm2.weight", "_orig_mod.model.layers.1.blocks.0.norm2.bias", "_orig_mod.model.layers.1.blocks.0.mlp.fc1.weight", "_orig_mod.model.layers.1.blocks.0.mlp.fc1.bias", "_orig_mod.model.layers.1.blocks.0.mlp.fc2.weight", "_orig_mod.model.layers.1.blocks.0.mlp.fc2.bias", "_orig_mod.model.layers.1.blocks.1.norm1.weight", "_orig_mod.model.layers.1.blocks.1.norm1.bias", "_orig_mod.model.layers.1.blocks.1.attn.relative_position_bias_table", "_orig_mod.model.layers.1.blocks.1.attn.qkv.weight", "_orig_mod.model.layers.1.blocks.1.attn.qkv.bias", "_orig_mod.model.layers.1.blocks.1.attn.proj.weight", "_orig_mod.model.layers.1.blocks.1.attn.proj.bias", "_orig_mod.model.layers.1.blocks.1.norm2.weight", "_orig_mod.model.layers.1.blocks.1.norm2.bias", "_orig_mod.model.layers.1.blocks.1.mlp.fc1.weight", "_orig_mod.model.layers.1.blocks.1.mlp.fc1.bias", "_orig_mod.model.layers.1.blocks.1.mlp.fc2.weight", "_orig_mod.model.layers.1.blocks.1.mlp.fc2.bias", "_orig_mod.model.layers.2.downsample.norm.weight", "_orig_mod.model.layers.2.downsample.norm.bias", "_orig_mod.model.layers.2.downsample.reduction.weight", "_orig_mod.model.layers.2.blocks.0.norm1.weight", "_orig_mod.model.layers.2.blocks.0.norm1.bias", "_orig_mod.model.layers.2.blocks.0.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.0.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.0.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.0.attn.proj.weight", "_orig_mod.model.layers.2.blocks.0.attn.proj.bias", "_orig_mod.model.layers.2.blocks.0.norm2.weight", "_orig_mod.model.layers.2.blocks.0.norm2.bias", "_orig_mod.model.layers.2.blocks.0.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.0.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.0.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.0.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.1.norm1.weight", "_orig_mod.model.layers.2.blocks.1.norm1.bias", "_orig_mod.model.layers.2.blocks.1.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.1.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.1.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.1.attn.proj.weight", "_orig_mod.model.layers.2.blocks.1.attn.proj.bias", "_orig_mod.model.layers.2.blocks.1.norm2.weight", "_orig_mod.model.layers.2.blocks.1.norm2.bias", "_orig_mod.model.layers.2.blocks.1.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.1.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.1.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.1.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.2.norm1.weight", "_orig_mod.model.layers.2.blocks.2.norm1.bias", "_orig_mod.model.layers.2.blocks.2.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.2.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.2.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.2.attn.proj.weight", "_orig_mod.model.layers.2.blocks.2.attn.proj.bias", "_orig_mod.model.layers.2.blocks.2.norm2.weight", "_orig_mod.model.layers.2.blocks.2.norm2.bias", "_orig_mod.model.layers.2.blocks.2.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.2.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.2.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.2.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.3.norm1.weight", "_orig_mod.model.layers.2.blocks.3.norm1.bias", "_orig_mod.model.layers.2.blocks.3.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.3.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.3.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.3.attn.proj.weight", "_orig_mod.model.layers.2.blocks.3.attn.proj.bias", "_orig_mod.model.layers.2.blocks.3.norm2.weight", "_orig_mod.model.layers.2.blocks.3.norm2.bias", "_orig_mod.model.layers.2.blocks.3.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.3.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.3.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.3.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.4.norm1.weight", "_orig_mod.model.layers.2.blocks.4.norm1.bias", "_orig_mod.model.layers.2.blocks.4.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.4.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.4.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.4.attn.proj.weight", "_orig_mod.model.layers.2.blocks.4.attn.proj.bias", "_orig_mod.model.layers.2.blocks.4.norm2.weight", "_orig_mod.model.layers.2.blocks.4.norm2.bias", "_orig_mod.model.layers.2.blocks.4.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.4.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.4.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.4.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.5.norm1.weight", "_orig_mod.model.layers.2.blocks.5.norm1.bias", "_orig_mod.model.layers.2.blocks.5.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.5.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.5.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.5.attn.proj.weight", "_orig_mod.model.layers.2.blocks.5.attn.proj.bias", "_orig_mod.model.layers.2.blocks.5.norm2.weight", "_orig_mod.model.layers.2.blocks.5.norm2.bias", "_orig_mod.model.layers.2.blocks.5.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.5.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.5.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.5.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.6.norm1.weight", "_orig_mod.model.layers.2.blocks.6.norm1.bias", "_orig_mod.model.layers.2.blocks.6.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.6.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.6.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.6.attn.proj.weight", "_orig_mod.model.layers.2.blocks.6.attn.proj.bias", "_orig_mod.model.layers.2.blocks.6.norm2.weight", "_orig_mod.model.layers.2.blocks.6.norm2.bias", "_orig_mod.model.layers.2.blocks.6.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.6.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.6.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.6.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.7.norm1.weight", "_orig_mod.model.layers.2.blocks.7.norm1.bias", "_orig_mod.model.layers.2.blocks.7.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.7.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.7.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.7.attn.proj.weight", "_orig_mod.model.layers.2.blocks.7.attn.proj.bias", "_orig_mod.model.layers.2.blocks.7.norm2.weight", "_orig_mod.model.layers.2.blocks.7.norm2.bias", "_orig_mod.model.layers.2.blocks.7.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.7.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.7.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.7.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.8.norm1.weight", "_orig_mod.model.layers.2.blocks.8.norm1.bias", "_orig_mod.model.layers.2.blocks.8.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.8.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.8.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.8.attn.proj.weight", "_orig_mod.model.layers.2.blocks.8.attn.proj.bias", "_orig_mod.model.layers.2.blocks.8.norm2.weight", "_orig_mod.model.layers.2.blocks.8.norm2.bias", "_orig_mod.model.layers.2.blocks.8.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.8.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.8.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.8.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.9.norm1.weight", "_orig_mod.model.layers.2.blocks.9.norm1.bias", "_orig_mod.model.layers.2.blocks.9.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.9.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.9.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.9.attn.proj.weight", "_orig_mod.model.layers.2.blocks.9.attn.proj.bias", "_orig_mod.model.layers.2.blocks.9.norm2.weight", "_orig_mod.model.layers.2.blocks.9.norm2.bias", "_orig_mod.model.layers.2.blocks.9.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.9.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.9.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.9.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.10.norm1.weight", "_orig_mod.model.layers.2.blocks.10.norm1.bias", "_orig_mod.model.layers.2.blocks.10.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.10.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.10.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.10.attn.proj.weight", "_orig_mod.model.layers.2.blocks.10.attn.proj.bias", "_orig_mod.model.layers.2.blocks.10.norm2.weight", "_orig_mod.model.layers.2.blocks.10.norm2.bias", "_orig_mod.model.layers.2.blocks.10.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.10.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.10.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.10.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.11.norm1.weight", "_orig_mod.model.layers.2.blocks.11.norm1.bias", "_orig_mod.model.layers.2.blocks.11.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.11.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.11.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.11.attn.proj.weight", "_orig_mod.model.layers.2.blocks.11.attn.proj.bias", "_orig_mod.model.layers.2.blocks.11.norm2.weight", "_orig_mod.model.layers.2.blocks.11.norm2.bias", "_orig_mod.model.layers.2.blocks.11.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.11.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.11.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.11.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.12.norm1.weight", "_orig_mod.model.layers.2.blocks.12.norm1.bias", "_orig_mod.model.layers.2.blocks.12.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.12.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.12.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.12.attn.proj.weight", "_orig_mod.model.layers.2.blocks.12.attn.proj.bias", "_orig_mod.model.layers.2.blocks.12.norm2.weight", "_orig_mod.model.layers.2.blocks.12.norm2.bias", "_orig_mod.model.layers.2.blocks.12.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.12.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.12.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.12.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.13.norm1.weight", "_orig_mod.model.layers.2.blocks.13.norm1.bias", "_orig_mod.model.layers.2.blocks.13.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.13.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.13.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.13.attn.proj.weight", "_orig_mod.model.layers.2.blocks.13.attn.proj.bias", "_orig_mod.model.layers.2.blocks.13.norm2.weight", "_orig_mod.model.layers.2.blocks.13.norm2.bias", "_orig_mod.model.layers.2.blocks.13.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.13.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.13.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.13.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.14.norm1.weight", "_orig_mod.model.layers.2.blocks.14.norm1.bias", "_orig_mod.model.layers.2.blocks.14.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.14.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.14.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.14.attn.proj.weight", "_orig_mod.model.layers.2.blocks.14.attn.proj.bias", "_orig_mod.model.layers.2.blocks.14.norm2.weight", "_orig_mod.model.layers.2.blocks.14.norm2.bias", "_orig_mod.model.layers.2.blocks.14.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.14.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.14.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.14.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.15.norm1.weight", "_orig_mod.model.layers.2.blocks.15.norm1.bias", "_orig_mod.model.layers.2.blocks.15.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.15.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.15.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.15.attn.proj.weight", "_orig_mod.model.layers.2.blocks.15.attn.proj.bias", "_orig_mod.model.layers.2.blocks.15.norm2.weight", "_orig_mod.model.layers.2.blocks.15.norm2.bias", "_orig_mod.model.layers.2.blocks.15.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.15.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.15.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.15.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.16.norm1.weight", "_orig_mod.model.layers.2.blocks.16.norm1.bias", "_orig_mod.model.layers.2.blocks.16.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.16.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.16.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.16.attn.proj.weight", "_orig_mod.model.layers.2.blocks.16.attn.proj.bias", "_orig_mod.model.layers.2.blocks.16.norm2.weight", "_orig_mod.model.layers.2.blocks.16.norm2.bias", "_orig_mod.model.layers.2.blocks.16.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.16.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.16.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.16.mlp.fc2.bias", "_orig_mod.model.layers.2.blocks.17.norm1.weight", "_orig_mod.model.layers.2.blocks.17.norm1.bias", "_orig_mod.model.layers.2.blocks.17.attn.relative_position_bias_table", "_orig_mod.model.layers.2.blocks.17.attn.qkv.weight", "_orig_mod.model.layers.2.blocks.17.attn.qkv.bias", "_orig_mod.model.layers.2.blocks.17.attn.proj.weight", "_orig_mod.model.layers.2.blocks.17.attn.proj.bias", "_orig_mod.model.layers.2.blocks.17.norm2.weight", "_orig_mod.model.layers.2.blocks.17.norm2.bias", "_orig_mod.model.layers.2.blocks.17.mlp.fc1.weight", "_orig_mod.model.layers.2.blocks.17.mlp.fc1.bias", "_orig_mod.model.layers.2.blocks.17.mlp.fc2.weight", "_orig_mod.model.layers.2.blocks.17.mlp.fc2.bias", "_orig_mod.model.layers.3.downsample.norm.weight", "_orig_mod.model.layers.3.downsample.norm.bias", "_orig_mod.model.layers.3.downsample.reduction.weight", "_orig_mod.model.layers.3.blocks.0.norm1.weight", "_orig_mod.model.layers.3.blocks.0.norm1.bias", "_orig_mod.model.layers.3.blocks.0.attn.relative_position_bias_table", "_orig_mod.model.layers.3.blocks.0.attn.qkv.weight", "_orig_mod.model.layers.3.blocks.0.attn.qkv.bias", "_orig_mod.model.layers.3.blocks.0.attn.proj.weight", "_orig_mod.model.layers.3.blocks.0.attn.proj.bias", "_orig_mod.model.layers.3.blocks.0.norm2.weight", "_orig_mod.model.layers.3.blocks.0.norm2.bias", "_orig_mod.model.layers.3.blocks.0.mlp.fc1.weight", "_orig_mod.model.layers.3.blocks.0.mlp.fc1.bias", "_orig_mod.model.layers.3.blocks.0.mlp.fc2.weight", "_orig_mod.model.layers.3.blocks.0.mlp.fc2.bias", "_orig_mod.model.layers.3.blocks.1.norm1.weight", "_orig_mod.model.layers.3.blocks.1.norm1.bias", "_orig_mod.model.layers.3.blocks.1.attn.relative_position_bias_table", "_orig_mod.model.layers.3.blocks.1.attn.qkv.weight", "_orig_mod.model.layers.3.blocks.1.attn.qkv.bias", "_orig_mod.model.layers.3.blocks.1.attn.proj.weight", "_orig_mod.model.layers.3.blocks.1.attn.proj.bias", "_orig_mod.model.layers.3.blocks.1.norm2.weight", "_orig_mod.model.layers.3.blocks.1.norm2.bias", "_orig_mod.model.layers.3.blocks.1.mlp.fc1.weight", "_orig_mod.model.layers.3.blocks.1.mlp.fc1.bias", "_orig_mod.model.layers.3.blocks.1.mlp.fc2.weight", "_orig_mod.model.layers.3.blocks.1.mlp.fc2.bias", "_orig_mod.model.norm.weight", "_orig_mod.model.norm.bias", "_orig_mod.head.weight", "_orig_mod.head.bias".

I think the issue may related with hyperparameters, but I dont sure.
It works well whitout hyperparameters.

**To Reproduce**

Train
``` python
from autogluon.multimodal import MultiModalPredictor
import uuid
import pandas as pd

train_data = pd.read_csv("train_list.txt",sep=" ")
model_path = f"./best_q_mixup_aug_2"
predictor = MultiModalPredictor(label="label", path=model_path,eval_metric="f1_macro",presets="best_quality").fit(
train_data=train_data,
hyperparameters={"env.compile.turn_on": True,"data.mixup.turn_on": True,"model.timm_image.train_transforms": ["resize_shorter_side", "center_crop", "trivial_augment","random_resize_crop", "random_horizontal_flip","color_jitter","randaug"]}
)
```
Load
``` python
from autogluon.multimodal import MultiModalPredictor
predictor = MultiModalPredictor.load("/root/cs/best_q_mixup_aug_1")
```

**Screenshots / Logs**

root@autodl-container-04e64d9cba-623e3541:~/autodl-tmp/cs# python fit.py
=================== System Info ===================
AutoGluon Version: 1.1.1
Python Version: 3.10.8
Operating System: Linux
Platform Machine: x86_64
Platform Version: #96-Ubuntu SMP Wed Sep 20 08:23:49 UTC 2023
CPU Count: 128
Pytorch Version: 2.3.1+cu118
CUDA Version: 11.8
Memory Avail: 868.38 GB / 1007.51 GB (86.2%)
Disk Space Avail: 29.04 GB / 50.00 GB (58.1%)
===================================================
AutoGluon infers your prediction problem is: 'multiclass' (because dtype of label-column == object).
5 unique label values: ['Vascular', 'Tumor', 'Inflammation', 'Cyst', 'Anomalies']
If 'multiclass' is not the correct problem_type, please manually specify the problem_type parameter during Predictor init (You may specify problem_type as one of: ['binary', 'multiclass', 'regression', 'quantile'])

AutoMM starts to create your model. ✨✨✨

To track the learning progress, you can open a terminal and launch Tensorboard:
```shell
# Assume you have installed tensorboard
tensorboard --logdir /root/cs/best_q_mixup_aug_1
```

Seed set to 0
GPU Count: 1
GPU Count to be Used: 1
GPU 0 Name: NVIDIA GeForce RTX 4090
GPU 0 Memory: 0.81GB/23.99GB (Used/Total)

Using 16bit Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
You are using a CUDA device ('NVIDIA GeForce RTX 4090') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

| Name | Type | Params | Mode
----------------------------------------------------------------
0 | model | OptimizedModule | 195 M | train
1 | validation_metric | MulticlassF1Score | 0 | train
2 | loss_func | CrossEntropyLoss | 0 | train
----------------------------------------------------------------
195 M Trainable params
0 Non-trainable params
195 M Total params
780.013 Total estimated model params size (MB)
Epoch 0: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [01:43<01:43, 3.81it/s]Epoch 0, global step 3: 'val_f1_macro' reached 0.07085 (best 0.07085), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=0-step=3.ckpt' as top 3
Epoch 0: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [02:18<00:00, 5.68it/s]Epoch 0, global step 6: 'val_f1_macro' reached 0.34331 (best 0.34331), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=0-step=6.ckpt' as top 3
Epoch 1: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [00:27<00:27, 14.20it/s]Epoch 1, global step 10: 'val_f1_macro' reached 0.39766 (best 0.39766), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=1-step=10.ckpt' as top 3
Epoch 1: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:03<00:00, 12.39it/s]Epoch 1, global step 13: 'val_f1_macro' reached 0.58320 (best 0.58320), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=1-step=13.ckpt' as top 3
Epoch 2: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [00:28<00:28, 13.84it/s]Epoch 2, global step 17: 'val_f1_macro' reached 0.70245 (best 0.70245), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=2-step=17.ckpt' as top 3
Epoch 2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:04<00:00, 12.18it/s]Epoch 2, global step 20: 'val_f1_macro' reached 0.79120 (best 0.79120), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=2-step=20.ckpt' as top 3
Epoch 3: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [00:27<00:27, 14.15it/s]Epoch 3, global step 24: 'val_f1_macro' reached 0.82172 (best 0.82172), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=3-step=24.ckpt' as top 3
Epoch 3: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:06<00:00, 11.91it/s]Epoch 3, global step 27: 'val_f1_macro' reached 0.82666 (best 0.82666), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=3-step=27.ckpt' as top 3
Epoch 4: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [00:27<00:27, 14.40it/s]Epoch 4, global step 31: 'val_f1_macro' reached 0.89469 (best 0.89469), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=4-step=31.ckpt' as top 3
Epoch 4: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:05<00:00, 12.12it/s]Epoch 4, global step 34: 'val_f1_macro' reached 0.87159 (best 0.89469), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=4-step=34.ckpt' as top 3
Epoch 5: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [00:28<00:28, 14.00it/s]Epoch 5, global step 38: 'val_f1_macro' reached 0.92111 (best 0.92111), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=5-step=38.ckpt' as top 3
Epoch 5: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:06<00:00, 11.88it/s]Epoch 5, global step 41: 'val_f1_macro' reached 0.88477 (best 0.92111), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=5-step=41.ckpt' as top 3
Epoch 6: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [00:28<00:28, 13.82it/s]Epoch 6, global step 45: 'val_f1_macro' reached 0.90564 (best 0.92111), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=6-step=45.ckpt' as top 3
Epoch 6: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:05<00:00, 12.09it/s]Epoch 6, global step 48: 'val_f1_macro' reached 0.94638 (best 0.94638), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=6-step=48.ckpt' as top 3
Epoch 7: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [00:27<00:27, 14.14it/s]Epoch 7, global step 52: 'val_f1_macro' reached 0.97333 (best 0.97333), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=7-step=52.ckpt' as top 3
Epoch 7: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:03<00:00, 12.45it/s]Epoch 7, global step 55: 'val_f1_macro' reached 0.98357 (best 0.98357), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=7-step=55.ckpt' as top 3
Epoch 8: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [00:28<00:28, 13.98it/s]Epoch 8, global step 59: 'val_f1_macro' reached 0.96905 (best 0.98357), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=8-step=59.ckpt' as top 3
Epoch 8: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:05<00:00, 12.05it/s]Epoch 8, global step 62: 'val_f1_macro' reached 0.98357 (best 0.98357), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=8-step=62.ckpt' as top 3
Epoch 9: 50%|██████████████████████████████████████████████████████████████████████████████▍ | 395/791 [00:26<00:26, 14.69it/s]Epoch 9, global step 66: 'val_f1_macro' reached 0.97801 (best 0.98357), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=9-step=66.ckpt' as top 3
Epoch 9: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:04<00:00, 12.34it/s]Epoch 9, global step 69: 'val_f1_macro' was not in top 3
Epoch 10: 50%|█████████████████████████████████████████████████████████████████████████████▉ | 395/791 [00:27<00:27, 14.40it/s]Epoch 10, global step 73: 'val_f1_macro' reached 0.98357 (best 0.98357), saving model to '/root/autodl-tmp/cs/best_q_mixup_aug_1/epoch=10-step=73.ckpt' as top 3
Epoch 10: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:03<00:00, 12.51it/s]Epoch 10, global step 76: 'val_f1_macro' was not in top 3
Epoch 11: 50%|█████████████████████████████████████████████████████████████████████████████▉ | 395/791 [00:28<00:28, 13.88it/s]Epoch 11, global step 80: 'val_f1_macro' was not in top 3
Epoch 11: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:02<00:00, 12.73it/s]Epoch 11, global step 83: 'val_f1_macro' was not in top 3
Epoch 12: 50%|█████████████████████████████████████████████████████████████████████████████▉ | 395/791 [00:28<00:28, 13.77it/s]Epoch 12, global step 87: 'val_f1_macro' was not in top 3
Epoch 12: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:01<00:00, 12.89it/s]Epoch 12, global step 90: 'val_f1_macro' was not in top 3
Epoch 12: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 790/791 [01:06<00:00, 11.91it/s]
Start to fuse 3 checkpoints via the greedy soup algorithm.
Predicting DataLoader 0: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [01:09<00:00, 0.72it/s]
Predicting DataLoader 0: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:00<00:00, 51.99it/s]
Predicting DataLoader 0: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:00<00:00, 65.63it/s]
AutoMM has created your model. 🎉🎉🎉

To load the model, use the code below:
```python
from autogluon.multimodal import MultiModalPredictor
predictor = MultiModalPredictor.load("/root/cs/best_q_mixup_aug_1")
```

If you are not satisfied with the model, try to increase the training time,
adjust the hyperparameters (https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/customization.html),
or post issues on GitHub (https://github.com/autogluon/autogluon/issues).

**Installed Versions**

INSTALLED VERSIONS
------------------
date : 2024-08-28
time : 08:46:43.708740
python : 3.10.8.final.0
OS : Linux
OS-release : 5.15.0-86-generic
Version : #96-Ubuntu SMP Wed Sep 20 08:23:49 UTC 2023
machine : x86_64
processor : x86_64
num_cores : 128
cpu_ram_mb : 1031691.21484375
cuda version : 12.535.129.03
num_gpus : 1
gpu_ram_mb : [19010]
avail_disk_size_mb : 20066

accelerate : 0.21.0
autogluon : 1.1.1
autogluon.common : 1.1.1
autogluon.core : 1.1.1
autogluon.features : 1.1.1
autogluon.multimodal : 1.1.1
autogluon.tabular : 1.1.1
autogluon.timeseries : 1.1.1
boto3 : 1.35.7
catboost : 1.2.5
defusedxml : 0.7.1
evaluate : 0.4.2
fastai : 2.7.17
gluonts : 0.15.1
hyperopt : 0.2.7
imodels : None
jinja2 : 3.1.2
joblib : 1.4.2
jsonschema : 4.21.1
lightgbm : 4.3.0
lightning : 2.3.3
matplotlib : 3.6.3
mlforecast : 0.10.0
networkx : 3.2.1
nlpaug : 1.1.11
nltk : 3.9.1
nptyping : 2.4.1
numpy : 1.24.1
nvidia-ml-py3 : 7.352.0
omegaconf : 2.2.3
onnxruntime-gpu : None
openmim : 0.3.9
optimum : 1.18.1
optimum-intel : None
orjson : 3.10.7
pandas : 2.2.2
pdf2image : 1.17.0
Pillow : 10.4.0
psutil : 5.9.4
pytesseract : 0.3.10
pytorch-lightning : 2.3.3
pytorch-metric-learning: 2.3.0
ray : 2.10.0
requests : 2.32.3
scikit-image : 0.20.0
scikit-learn : 1.4.0
scikit-learn-intelex : None
scipy : 1.12.0
seqeval : 1.2.2
setuptools : 65.5.0
skl2onnx : None
statsforecast : 1.4.0
tabpfn : None
tensorboard : 2.11.2
text-unidecode : 1.3
timm : 0.9.16
torch : 2.3.1+cu118
torchmetrics : 1.2.1
torchvision : 0.18.1+cu118
tqdm : 4.66.5
transformers : 4.39.3
utilsforecast : 0.0.10
vowpalwabbit : None
xgboost : 2.0.3

Start with the minimal image-classification example and the MultiModalPredictor.load entry point shown in autogluon/multimodal/predictor.py. Trace BaseLearner.load and BaseLearner._load_state_dict in autogluon/multimodal/learners/base.py, focusing on the reported TimmAutoModelForImagePrediction state-dict mismatch. Done means a trained predictor can be loaded successfully without the listed missing-key error.

Written by the indexing model from the issue text.