Experiment should merge redundant feature group subsets
- Dominant language
- Jupyter Notebook
- Stars
- 201
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
If the user creates a bunch of feature group subsets, there can be some redundancy. We should do some mixture of:
- blowing up in the experiment validator if they do configuration that makes no sense (like mixing all-combinations with anything else)
- merging redundant subsets if they are arrived at through sane config (e.g. leave-one-in and leave-one-out overlap)
Contributor guide
Research direction
Start by locating the experiment validator and the code that constructs feature-group subsets. Determine which configurations are invalid, including mixing all-combinations with other modes, and how leave-one-in and leave-one-out overlap. Done means invalid configurations fail validation and valid redundant subsets are merged, with tests covering both cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100