facebook / facebook/prophet

Stream data - refitting model results in error for 1 data point

Open
#2,208 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
20.4k
Forks
4.6k
Avg merge
19h 52m
Merged PRs (30d)
1

Description

The use case is we're setting up prophet on IoT real-time forecasting/anomaly detection.

The plan is to "incept" the model on historical data (around the last 55_000 data points)
Serialize the model to storage.

Now run through this process on a stream of data.

1. Download model from blob.
2. Bootstrap model using refitting documentation
3. Ask for prediction for next data point ```make_future_dataframe(periods=1, freq='15Min')```
4. Is the data point coming in inside the bounds? if yes anomaly if no not anomaly.
5. Add the data point to data frame, add dataframe to existing model (using re-fitting documentation)
6. Save model back to blob.

After setting this process up we're seeing an error "Dataframe has less than 2 non-NaN rows" which upon reading through some stuff online looks like an error but the model has been trained on 55_000 data points all of which are stored inside the serailized_model.json file, could this be change to not error if the model is being refitted?

(pseudo code that causes error)
```py
df1 = pd.DataFrame([json.loads('{"ds": "2022-06-27 20:50:30", "y": "20.5"}')])
df1['ds'] = pd.to_datetime(df1['ds'])
df1 = df1.sort_values('ds')
df1 = df1.set_index('ds').resample('15Min', origin='epoch').mean().ffill().reset_index()
model = Prophet().fit(df1, init=stan_init(model_from_json(mod)))
```
Or is there another methodology you would recommend to run prophet on streaming data after an initial training session?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.