automl / automl/DeepCAVE

[Feature Request] HP Interactions in fANOVA

Open
#149 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
104
Forks
15
PR merge metrics
No merged PRs in 30d

Description

In principle, fANONVA can compute pariwise/setwise marginals to inspect hp interactions. In the current fANOVA evaluator, all of this is disabled, but actually it would just need some re-parsing of the existing output in the disabled functions. For pairwise marginals, I did this here: https://github.com/automl/arlbench/blob/experiments/rs_data_analysis/fanova_interactions.ipynb

If you compare that code to the current fANOVA evaluator, you'll see that's it's really only a matter of using hp names vs indices and adding the depth parameter to the importance call. How to plot this well is maybe another matter, but basic integration could be quite easy, e.g. via an option in process to also compute the marginals as it's really only one function call.

Contributor guide

No contributing guide indexed for this repository

Research direction

Compare the existing fANOVA evaluator with the referenced arlbench experiments/rs_data_analysis/fanova_interactions.ipynb, focusing on the disabled interaction functions and their current output parsing. Verify how hyperparameter names, indices, and the depth argument are handled; the work is done when process can optionally compute pairwise or setwise marginals, with the expected interaction data available for plotting.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.