continuous training with steaming data
- Dominant language
- Python
- Stars
- 20.4k
- Forks
- 4.6k
- Avg merge
- 19h 52m
- Merged PRs (30d)
- 1
Description
I have a use case of streaming data. Let's say I have seasonalities like time of the day, day of the week etc. and I have streaming data coming in on which I want to train as well as forecast (and compare forecast against actual values that come in later). I was thinking that I could train a model, say for a week. I can use it to make predictions for the next two hours against which I can compare my actual values during that time. At the same time I can collect hourly data and keep incrementally training my model for continuous improvements.
I came across https://github.com/facebook/prophet/issues/46 which could have helped here. However I do not find any 'partial_fit' functionality in prophet though the issue is closed.
From some comments and posting I gathered some code with stan_init and I put together https://github.com/sameermahajan/MLWorkshop/tree/master/15.%20Timeseries/incremental with complete code and sample data to try out. However while trying that I ran into following error. Isn't it generic? What am I missing?
Unrecoverable error evaluating the log probability at the initial value.
mismatch in dimension declared and found in context; processing stage=parameter initialization; variable name=delta; position=0; dims declared=(1); dims found=(8)
WARNING:fbprophet.models:Optimization terminated abnormally. Falling back to Newton.
Unrecoverable error evaluating the log probability at the initial value.
mismatch in dimension declared and found in context; processing stage=parameter initialization; variable name=delta; position=0; dims declared=(1); dims found=(8)
Traceback (most recent call last):
File ".../Library/Python/3.9/lib/python/site-packages/fbprophet/models.py", line 245, in fit
self.stan_fit = self.model.optimizing(**args)
File ".../Library/Python/3.9/lib/python/site-packages/pystan/model.py", line 581, in optimizing
ret, sample = fit._call_sampler(stan_args)
File "stanfit4anon_model_dfdaf2b8ece8a02eb11f050ec701c0ec_3960193224443467293.pyx", line 1046, in stanfit4anon_model_dfdaf2b8ece8a02eb11f050ec701c0ec_3960193224443467293.StanFit4Model._call_sampler
File "stanfit4anon_model_dfdaf2b8ece8a02eb11f050ec701c0ec_3960193224443467293.pyx", line 406, in stanfit4anon_model_dfdaf2b8ece8a02eb11f050ec701c0ec_3960193224443467293._call_sampler
RuntimeError: mismatch in dimension declared and found in context; processing stage=parameter initialization; variable name=delta; position=0; dims declared=(1); dims found=(8)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File ".../aerust2/aerust/aeris/aeris-seasonality/examples/incremental_training.py", line 17, in
m2.fit(df, init=stan_init2)
File ".../Library/Python/3.9/lib/python/site-packages/fbprophet/forecaster.py", line 1166, in fit
self.params = self.stan_backend.fit(stan_init, dat, **kwargs)
File ".../Library/Python/3.9/lib/python/site-packages/fbprophet/models.py", line 252, in fit
self.stan_fit = self.model.optimizing(**args)
File ".../Library/Python/3.9/lib/python/site-packages/pystan/model.py", line 581, in optimizing
ret, sample = fit._call_sampler(stan_args)
File "stanfit4anon_model_dfdaf2b8ece8a02eb11f050ec701c0ec_3960193224443467293.pyx", line 1046, in stanfit4anon_model_dfdaf2b8ece8a02eb11f050ec701c0ec_3960193224443467293.StanFit4Model._call_sampler
File "stanfit4anon_model_dfdaf2b8ece8a02eb11f050ec701c0ec_3960193224443467293.pyx", line 406, in stanfit4anon_model_dfdaf2b8ece8a02eb11f050ec701c0ec_3960193224443467293._call_sampler
RuntimeError: mismatch in dimension declared and found in context; processing stage=parameter initialization; variable name=delta; position=0; dims declared=(1); dims found=(8)
Contributor guide
Research direction
Start with the incremental_training.py example and sample data linked in the issue, then inspect fit handling in fbprophet/forecaster.py and fbprophet/models.py. Reproduce the delta dimension mismatch and determine whether continuous or incremental training is intended to be supported. Done should mean a decided, documented behavior or a tested implementation for the reported use case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100