Missing data allowed when training, but not with tree interpreters
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 4.8k
- Forks
- 827
- PR merge metrics
- No merged PRs in 30d
Description
This issue is self-explanatory.
Objects like DML have an allow_missing parameter that allows null values as an input.
This can work fine in two scenarios:
- when the underlying models can deal with missing values
- when a feature preprocessor is used to deal with nulls before the data is fed to the underlying models
In both those scenarios, it's possible to train DML, but when trying to use a tree interpreter, those use X directly for both the tree and the DML predictions. It does not leverage the feature preprocessor used by the DML object, just the raw X matrix.
Ideally, we'd allow a separate way to deal with null values for the tree interpreters, so they are not completely unavailable when training DML with null values.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the DML tree-interpreter paths and the handling of the allow_missing parameter described in the issue. Trace how raw X bypasses the DML feature preprocessor, then determine how a separate missing-value strategy could be supplied to tree interpreters. Done means DML trained with missing values can be interpreted without the current restriction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100