New plugin: Hardness of AutoML problem
- Dominant language
- Python
- Stars
- 104
- Forks
- 15
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
We often wonder how hard an AutoML problem is. Can we therefore add some metrics regarding that?
For example
* an eCDF plot for the cost distribution (i.e., a hard AutoML task should have only a few very well-performing configurations)
* uni-modal metric from the automl loss landscape paper (but evaluated on our surrogate models)
* convexity metric from the automl loss landscape paper (but evaluated on our surrogate models)
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are identified. Start by locating the surrogate-model analysis and visualization components, then clarify the proposed cost-distribution, uni-modal, and convexity metrics and how completion would be evaluated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-visualization, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100