Dask Stream Closed Error with Bayesian Optimization
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
hi all,
I've been trying to create a back testing framework for a time series analysis that involves tuning a random forest with Bayesian optimization monthly for the last 10 years. each month is independent of all others so they can be tested at the same time so I've been trying to use dask to distribute all of the training/predictions. when run linearly without dask, the task completes with no errors. however with dask I get a stream closed error for some of the months. I've found the number of months that complete without error is inversely proportional to the number of iterations that Bayesian optimization runs (which essentially makes each month run longer). 60 Bayesian optimization iterations for each month resulted in 32/132 months being completed, and 30 iterations resulted in 64/132. the Bayesian optimization library I'm using is at https://github.com/fmfn/BayesianOptimization
does anyone have any idea why this might be happening? Is this an issue with dask, some config issue or with the BO library incompatibility?
Contributor guide
Assessment
This issue has not been assessed yet.