[Feature Request] HP Interactions in fANOVA
- Dominant language
- Python
- Stars
- 104
- Forks
- 15
- PR merge metrics
- No merged PRs in 30d
Description
In principle, fANONVA can compute pariwise/setwise marginals to inspect hp interactions. In the current fANOVA evaluator, all of this is disabled, but actually it would just need some re-parsing of the existing output in the disabled functions. For pairwise marginals, I did this here: https://github.com/automl/arlbench/blob/experiments/rs_data_analysis/fanova_interactions.ipynb
If you compare that code to the current fANOVA evaluator, you'll see that's it's really only a matter of using hp names vs indices and adding the depth parameter to the importance call. How to plot this well is maybe another matter, but basic integration could be quite easy, e.g. via an option in process to also compute the marginals as it's really only one function call.
Contributor guide
No contributing guide indexed for this repository
Research direction
Compare the existing fANOVA evaluator with the referenced arlbench experiments/rs_data_analysis/fanova_interactions.ipynb, focusing on the disabled interaction functions and their current output parsing. Verify how hyperparameter names, indices, and the depth argument are handled; the work is done when process can optionally compute pairwise or setwise marginals, with the expected interaction data available for plotting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-visualization, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100