autogluon / autogluon/autogluon
[BUG] Cannot Hyperparameter tune RecursiveTabular with TimeSeriesPredictor
- Dominant language
- Python
- Stars
- 10.7k
- Forks
- 1.2k
- Avg merge
- 21h 29m
- Merged PRs (30d)
- 57
Description
**Bug Report Checklist**
- [x] I provided code that demonstrates a minimal reproducible example.
- [ ] I confirmed bug exists on the latest mainline of AutoGluon via source install.
- [x] I confirmed bug exists on the latest stable version of AutoGluon.
**Describe the bug**
The documentation shows how to tune [DeepAR](https://auto.gluon.ai/stable/tutorials/timeseries/forecasting-indepth.html#hyperparameter-tuning) for TimeSeries, but doesn't show how to do RecursiveTabular. As such, I'm getting unexepcted messages given setups that seem reasonable.
```
predictor = TimeSeriesPredictor(
prediction_length=21,
freq='h',
)
predictor.fit(
train_data=train_data,
hyperparameters={
"DeepAR": {},
"RecursiveTabular": [
{"tabular_hyperparameters": {"GBM": { "num_leaves": space.Int(5, 50)}}}
],
},
hyperparameter_tune_kwargs={
"scheduler": "local",
"searcher": "auto",
"num_trials": 5,
},
enable_ensemble = False,
time_limit = 600
)
```
with this error:
```
ValueError: Hyperparameter tuning specified, but no model contains a hyperparameter search space. Please disable hyperparameter tuning with `hyperparameter_tune_kwargs=None` or provide a search space for at least one model.
```
**Expected behavior**
I'd expect this to work given how hyperparameters are defined in https://github.com/autogluon/autogluon/issues/2969, though the documentation isn't very clear at all.
**To Reproduce**
I've tried maybe 15 different ways of setting up the configs. I've poured through https://github.com/autogluon/autogluon/blob/579ede12d9157778d90c617c4bfddd0b4865a582/timeseries/src/autogluon/timeseries/models/presets.py#L314-L331 to understand what types of parameter combinations are expected.
I've also tried this:
```
"RecursiveTabular": [
{"GBM": { "num_leaves": space.Int(5, 50)}}
],
```
and I get the same errors. Trying this runs the HPO process, but is masked by the error from https://github.com/autogluon/autogluon/issues/2969
```
"RecursiveTabular": [
{ "num_leaves": space.Int(5, 50)}
],
```
**Installed Versions**
INSTALLED VERSIONS
------------------
date : 2024-07-13
time : 22:11:00.195083
python : 3.10.8.final.0
OS : Linux
OS-release : 5.10.215-203.850.amzn2.x86_64
Version : #1 SMP Tue Apr 23 20:32:19 UTC 2024
machine : x86_64
processor : x86_64
num_cores : 64
cpu_ram_mb : 255019.73828125
cuda version : 12.535.161.08
num_gpus : 1
gpu_ram_mb : [22365]
avail_disk_size_mb : 61511
accelerate : 0.21.0
autogluon : 1.1.1
autogluon.common : 1.1.1
autogluon.core : 1.1.1
autogluon.features : 1.1.1
autogluon.multimodal : 1.1.1
autogluon.tabular : 1.1.1
autogluon.timeseries : 1.1.1
boto3 : 1.34.101
catboost : 1.2.5
defusedxml : 0.7.1
evaluate : 0.4.2
fastai : 2.7.15
gluonts : 0.15.1
hyperopt : 0.2.7
imodels : None
jinja2 : 3.1.4
joblib : 1.4.2
jsonschema : 4.21.1
lightgbm : 4.3.0
lightning : 2.3.2
matplotlib : 3.9.1
mlforecast : 0.10.0
networkx : 3.3
nlpaug : 1.1.11
nltk : 3.8.1
nptyping : 2.4.1
numpy : 1.26.4
nvidia-ml-py3 : 7.352.0
omegaconf : 2.2.3
onnxruntime-gpu : None
openmim : 0.3.9
optimum : 1.18.1
optimum-intel : 1.16.1
orjson : 3.10.6
pandas : 2.2.2
pdf2image : 1.17.0
Pillow : 10.4.0
psutil : 5.9.8
pytesseract : 0.3.10
pytorch-lightning : 2.3.2
pytorch-metric-learning: 2.3.0
ray : 2.10.0
requests : 2.32.3
scikit-image : 0.20.0
scikit-learn : 1.4.0
scikit-learn-intelex : None
scipy : 1.12.0
seqeval : 1.2.2
setuptools : 69.5.1
skl2onnx : None
statsforecast : 1.4.0
tabpfn : None
tensorboard : 2.17.0
text-unidecode : 1.3
timm : 0.9.16
torch : 2.3.1
torchmetrics : 1.2.1
torchvision : 0.18.1
tqdm : 4.66.4
transformers : 4.39.3
utilsforecast : 0.0.10
vowpalwabbit : None
xgboost : 2.0.3
Contributor guide
Research direction
Start with timeseries/src/autogluon/timeseries/models/presets.py lines 314-331 and the documented DeepAR tuning example for TimeSeriesPredictor. Reproduce the RecursiveTabular configurations on AutoGluon 1.1.1, compare the behavior with issue #2969, and verify that the supported configuration or documentation no longer produces the reported ValueError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100