autogluon / autogluon/autogluon
[BUG] MultimodalPredictor Ensemble Fitting Error for Binary Classification
- Dominant language
- Python
- Stars
- 10.7k
- Forks
- 1.2k
- Avg merge
- 21h 29m
- Merged PRs (30d)
- 57
Description
**Bug Report Checklist**
- [ ] I provided code that demonstrates a minimal reproducible example.
- [x] I confirmed bug exists on the latest mainline of AutoGluon via source install.
- [x] I confirmed bug exists on the latest stable version of AutoGluon.
**Describe the bug**
whenever I try to use MultiModalPredictor with use_ensemble and fit() it onto a binary classification problem, I get a KeyError, mainly because of this line: https://github.com/autogluon/autogluon/blob/71360e1e363d07262d08fedef47692970ec093bb/multimodal/src/autogluon/multimodal/learners/ensemble.py#L461
where it expects Y_PRED_PROB instead of Y_PRED for classification tasks
**Expected behavior**
It should just fit() and metric_data should have Y_PREDICT_PROB as a key for classification tasks
**To Reproduce**
I use my own large dataset so it's pretty hard to upload both the dataset but I pretty use ensemble multimodalpredictor with mu
**Screenshots / Logs**
```
Traceback (most recent call last):
File "/home/idies/workspace/Temporary/akhot2/scratch/SUBMOD_SCI/Spatial-CI/space-data/train_spaceenv.py", line 286, in main
treat_predictor = treat_trainer.fit(
File "/home/idies/workspace/Temporary/akhot2/scratch/spacedata/lib/python3.10/site-packages/autogluon/multimodal/predictor.py", line 540, in fit
self._learner.fit(
File "/home/idies/workspace/Temporary/akhot2/scratch/spacedata/lib/python3.10/site-packages/autogluon/multimodal/learners/ensemble.py", line 510, in fit
weighted_ensemble, selected_learner_indices = self.fit_ensemble(
File "/home/idies/workspace/Temporary/akhot2/scratch/spacedata/lib/python3.10/site-packages/autogluon/multimodal/learners/ensemble.py", line 464, in fit_ensemble
score = compute_score(
File "/home/idies/workspace/Temporary/akhot2/scratch/spacedata/lib/python3.10/site-packages/autogluon/multimodal/optim/metrics/utils.py", line 100, in compute_score
y_pred_proba = metric_data[Y_PRED_PROB]
KeyError: 'y_pred_prob'
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
```
**Installed Versions**
```python
# Replace this code with the output of the following:
from autogluon.core.utils import show_versions
show_versions()
INSTALLED VERSIONS
------------------
date : 2025-08-06
time : 08:28:35.628948
python : 3.10.16.final.0
OS : Linux
OS-release : 5.14.0-570.19.1.el9_6.x86_64
Version : #1 SMP PREEMPT_DYNAMIC Wed Jun 4 04:00:24 EDT 2025
machine : x86_64
processor : x86_64
num_cores : 80
cpu_ram_mb : 773148.86328125
cuda version : 12.575.57.08
num_gpus : 4
gpu_ram_mb : [32495, 32495, 32495, 32495]
avail_disk_size_mb : 5594949938
accelerate : 1.9.0
autogluon : 1.4.0
autogluon.common : 1.4.0
autogluon.core : 1.4.0
autogluon.features : 1.4.0
autogluon.multimodal : 1.4.0
autogluon.tabular : 1.4.0
autogluon.timeseries : 1.4.0
blis : 1.2.1
boto3 : 1.37.10
catboost : 1.2.8
coreforecast : 0.0.16
defusedxml : 0.7.1
einops : None
einx : 0.3.0
evaluate : 0.4.5
fastai : 2.7.19
fsspec : 2025.3.0
fugue : 0.9.1
gluonts : 0.16.2
huggingface-hub : 0.34.3
hyperopt : 0.2.7
imodels : 1.3.18
jinja2 : 3.1.6
joblib : 1.5.1
jsonschema : 4.23.0
lightgbm : 4.6.0
lightning : 2.5.2
loguru : 0.7.3
matplotlib : 3.10.5
mlforecast : 0.14.0
networkx : 3.4.2
nlpaug : 1.1.11
nltk : 3.9.1
numpy : 1.26.4
nvidia-ml-py3 : 7.352.0
omegaconf : 2.3.0
onnx : None
onnxruntime : None
onnxruntime-gpu : None
openmim : 0.3.9
orjson : 3.10.18
pandas : 2.3.0
pdf2image : 1.17.0
Pillow : 11.2.1
psutil : 5.9.0
pyarrow : 20.0.0
pytabkit : None
pytesseract : 0.3.13
pytorch-lightning : 2.5.1.post0
pytorch-metric-learning: 2.8.1
ray : 2.44.1
requests : 2.32.3
scikit-image : 0.25.2
scikit-learn : 1.7.1
scikit-learn-intelex : None
scipy : 1.11.4
seqeval : 1.2.2
skl2onnx : None
spacy : 3.8.7
statsforecast : 2.0.1
tabicl : None
tabpfn : None
tensorboard : 2.20.0
text-unidecode : 1.3
timm : 1.0.3
torch : 2.6.0
torchmetrics : 1.7.3
torchvision : 0.21.0
tqdm : 4.67.1
transformers : 4.49.0
utilsforecast : 0.2.11
xgboost : 3.0.3
```
Contributor guide
Research direction
Start at multimodal/src/autogluon/multimodal/learners/ensemble.py around line 461 and follow the metric_data keys into multimodal/src/autogluon/multimodal/optim/metrics/utils.py, where the traceback shows the KeyError. Reproduce binary classification with MultiModalPredictor and use_ensemble, then verify fitting completes and classification metric_data contains the expected prediction-probability key.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100