dask / dask/distributed

Dask Stream Closed Error with Bayesian Optimization

Open
#807 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

hi all,

I've been trying to create a back testing framework for a time series analysis that involves tuning a random forest with Bayesian optimization monthly for the last 10 years. each month is independent of all others so they can be tested at the same time so I've been trying to use dask to distribute all of the training/predictions. when run linearly without dask, the task completes with no errors. however with dask I get a stream closed error for some of the months. I've found the number of months that complete without error is inversely proportional to the number of iterations that Bayesian optimization runs (which essentially makes each month run longer). 60 Bayesian optimization iterations for each month resulted in 32/132 months being completed, and 30 iterations resulted in 64/132. the Bayesian optimization library I'm using is at https://github.com/fmfn/BayesianOptimization
does anyone have any idea why this might be happening? Is this an issue with dask, some config issue or with the BO library incompatibility?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.