autogluon / autogluon/autogluon

[BUG] MultimodalPredictor Ensemble Fitting Error for Binary Classification

Open
#5,276 0 comments 0 reactions 0 assignees View on GitHub
bug: unconfirmed module: multimodal Needs Triage
Dominant language
Python
Stars
10.7k
Forks
1.2k
Avg merge
21h 29m
Merged PRs (30d)
57

Description

**Bug Report Checklist**

- [ ] I provided code that demonstrates a minimal reproducible example.
- [x] I confirmed bug exists on the latest mainline of AutoGluon via source install.
- [x] I confirmed bug exists on the latest stable version of AutoGluon.

**Describe the bug**

whenever I try to use MultiModalPredictor with use_ensemble and fit() it onto a binary classification problem, I get a KeyError, mainly because of this line: https://github.com/autogluon/autogluon/blob/71360e1e363d07262d08fedef47692970ec093bb/multimodal/src/autogluon/multimodal/learners/ensemble.py#L461

where it expects Y_PRED_PROB instead of Y_PRED for classification tasks

**Expected behavior**

It should just fit() and metric_data should have Y_PREDICT_PROB as a key for classification tasks

**To Reproduce**

I use my own large dataset so it's pretty hard to upload both the dataset but I pretty use ensemble multimodalpredictor with mu

**Screenshots / Logs**

```
Traceback (most recent call last):
File "/home/idies/workspace/Temporary/akhot2/scratch/SUBMOD_SCI/Spatial-CI/space-data/train_spaceenv.py", line 286, in main
treat_predictor = treat_trainer.fit(
File "/home/idies/workspace/Temporary/akhot2/scratch/spacedata/lib/python3.10/site-packages/autogluon/multimodal/predictor.py", line 540, in fit
self._learner.fit(
File "/home/idies/workspace/Temporary/akhot2/scratch/spacedata/lib/python3.10/site-packages/autogluon/multimodal/learners/ensemble.py", line 510, in fit
weighted_ensemble, selected_learner_indices = self.fit_ensemble(
File "/home/idies/workspace/Temporary/akhot2/scratch/spacedata/lib/python3.10/site-packages/autogluon/multimodal/learners/ensemble.py", line 464, in fit_ensemble
score = compute_score(
File "/home/idies/workspace/Temporary/akhot2/scratch/spacedata/lib/python3.10/site-packages/autogluon/multimodal/optim/metrics/utils.py", line 100, in compute_score
y_pred_proba = metric_data[Y_PRED_PROB]
KeyError: 'y_pred_prob'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
```

**Installed Versions**

```python
# Replace this code with the output of the following:
from autogluon.core.utils import show_versions
show_versions()
INSTALLED VERSIONS
------------------
date : 2025-08-06
time : 08:28:35.628948
python : 3.10.16.final.0
OS : Linux
OS-release : 5.14.0-570.19.1.el9_6.x86_64
Version : #1 SMP PREEMPT_DYNAMIC Wed Jun 4 04:00:24 EDT 2025
machine : x86_64
processor : x86_64
num_cores : 80
cpu_ram_mb : 773148.86328125
cuda version : 12.575.57.08
num_gpus : 4
gpu_ram_mb : [32495, 32495, 32495, 32495]
avail_disk_size_mb : 5594949938

accelerate : 1.9.0
autogluon : 1.4.0
autogluon.common : 1.4.0
autogluon.core : 1.4.0
autogluon.features : 1.4.0
autogluon.multimodal : 1.4.0
autogluon.tabular : 1.4.0
autogluon.timeseries : 1.4.0
blis : 1.2.1
boto3 : 1.37.10
catboost : 1.2.8
coreforecast : 0.0.16
defusedxml : 0.7.1
einops : None
einx : 0.3.0
evaluate : 0.4.5
fastai : 2.7.19
fsspec : 2025.3.0
fugue : 0.9.1
gluonts : 0.16.2
huggingface-hub : 0.34.3
hyperopt : 0.2.7
imodels : 1.3.18
jinja2 : 3.1.6
joblib : 1.5.1
jsonschema : 4.23.0
lightgbm : 4.6.0
lightning : 2.5.2
loguru : 0.7.3
matplotlib : 3.10.5
mlforecast : 0.14.0
networkx : 3.4.2
nlpaug : 1.1.11
nltk : 3.9.1
numpy : 1.26.4
nvidia-ml-py3 : 7.352.0
omegaconf : 2.3.0
onnx : None
onnxruntime : None
onnxruntime-gpu : None
openmim : 0.3.9
orjson : 3.10.18
pandas : 2.3.0
pdf2image : 1.17.0
Pillow : 11.2.1
psutil : 5.9.0
pyarrow : 20.0.0
pytabkit : None
pytesseract : 0.3.13
pytorch-lightning : 2.5.1.post0
pytorch-metric-learning: 2.8.1
ray : 2.44.1
requests : 2.32.3
scikit-image : 0.25.2
scikit-learn : 1.7.1
scikit-learn-intelex : None
scipy : 1.11.4
seqeval : 1.2.2
skl2onnx : None
spacy : 3.8.7
statsforecast : 2.0.1
tabicl : None
tabpfn : None
tensorboard : 2.20.0
text-unidecode : 1.3
timm : 1.0.3
torch : 2.6.0
torchmetrics : 1.7.3
torchvision : 0.21.0
tqdm : 4.67.1
transformers : 4.49.0
utilsforecast : 0.2.11
xgboost : 3.0.3
```

Contributor guide

Open the contributing guide

Research direction

Start at multimodal/src/autogluon/multimodal/learners/ensemble.py around line 461 and follow the metric_data keys into multimodal/src/autogluon/multimodal/optim/metrics/utils.py, where the traceback shows the KeyError. Reproduce binary classification with MultiModalPredictor and use_ensemble, then verify fitting completes and classification metric_data contains the expected prediction-probability key.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.