Should we consider adding bayesmark to the benchmarking suite
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 176
- Forks
- 40
- PR merge metrics
- No merged PRs in 30d
Description
The bayesmark package is another wrapper hyper parameter tuning library. We can add this to our benchmarking suite. Per their documentation, they wrap around:
The builtin optimizers are wrappers on the following projects:
HyperOpt
Nevergrad
OpenTuner
PySOT
Scikit-optimize
https://github.com/uber/bayesmark/
And we already benchmark against HyperOpt. Note that OpenTuner is a previous package developed at MIT in 2014.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing HyperOpt benchmark and the bayesmark documentation linked in the issue, including its wrapped optimizers. Determine how the current benchmarking suite incorporates HyperOpt; done means bayesmark is added in a comparable way and its benchmark results can be run alongside the existing suite.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100