facebook / facebook/prophet

Running cross validation causes IndexError: single positional indexer is out-of-bounds

Open
#2,371 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
20.4k
Forks
4.6k
Avg merge
19h 52m
Merged PRs (30d)
1

Description

I have a dataset where there is gaps in the data. Because of this, when I try to run the cross-validation on the data, the code fails with error `IndexError: single positional indexer is out-of-bounds` because I guess it cannot find any data on the test set to compare the performance against.

I have also posted this on stack overflow - https://stackoverflow.com/questions/75488566/how-to-handle-cross-validation-in-prophet-with-gaps-in-the-dataset

Here is my code -

```
from prophet.diagnostics import cross_validation as cv, performance_metrics as pm
import pandas as pd
df = pd.read_csv('..')
model = ProphetPos(holidays = holidays, **config.MODEL_PARAMS)
model.fit(df)
results = cv(model, f'182 days', initial=f'182', period='30 days', parallel='threads')
```

And here is the logs -
```

File "/usr/local/lib/python3.8/site-packages/prophet/diagnostics.py", line 202, in cross_validation
return pd.concat(predicts, axis=0).reset_index(drop=True)
File "/usr/local/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 285, in concat
op = _Concatenator(
File "/usr/local/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 339, in __init__
objs = list(objs)
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator
yield fs.pop().result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 444, in result
return self.__get_result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.8/site-packages/prophet/diagnostics.py", line 250, in single_cutoff_forecast
yhat = m.predict(df[index_predicted][columns])
File "/usr/local/lib/python3.8/site-packages/inf_forecast_colo/utils/prophet_forecaster.py", line 1682, in predict
if fcst['trend'].iloc[-1] < fcst['trend'].iloc[-2]:
File "/usr/local/lib/python3.8/site-packages/pandas/core/indexing.py", line 895, in __getitem__
return self._getitem_axis(maybe_callable, axis=axis)
File "/usr/local/lib/python3.8/site-packages/pandas/core/indexing.py", line 1501, in _getitem_axis
self._validate_integer(key, axis)
File "/usr/local/lib/python3.8/site-packages/pandas/core/indexing.py", line 1444, in _validate_integer
raise IndexError("single positional indexer is out-of-bounds")
IndexError: single positional indexer is out-of-bounds
```

Is there a way to efficiently handle the gaps in the data and still be able to do cross validation?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.