Add ability to see if a base feature will have an impact on predictions
- Dominant language
- Python
- Stars
- 850
- Forks
- 96
- PR merge metrics
- No merged PRs in 30d
Description
- As a user, I wish I could easily see whether a feature in X will be passed to the estimator in a pipeline.
While implementing #3753, it became clear that there are times when you can look at a fit pipeline and know whether or not a feature in X will have an impact on predictions by determining if the feature is ultimately passed to the estimator. If a feature isn't in the transformed dataset produced by `pipeline.transform_all_but_final`, then it's not passed to the estimator and therefore has no impact on predictions! Such features should have a feature importance of 0 and a partial dependence that is the same for all feature values. Those can be lengthy calculations, but we already have the information that those measures would tell us, so we shouldn't need to make those calculations.
If we had an easy way to see this information, it could be leveraged to avoid unnecessary computation!
Situations where this may happen:
- RF Regressor/Classifier Feature Selector determines that a feature will be unimportant and drops it
- `DropColumns` transformer is used to drop specific columns
- `SelectColumns` transformer is used and leaves some columns out
#### Code Example
A potential user-facing api:
```python
pipeline.is_feature_impactful(feature_name)
```
The implementation
```python
def is_feature_impactful(self, X, feature_name)
X_t = self.transform_all_but_final(X)
feature_provenance = self._get_feature_provenance()
return feature_name in feature_provenance or feature name in X_t.columns
```
Note that doing the `pipeline.transform_all_but_final(X)` calculation over and over for every feature is unnecessary repeted computation. So we could either allow passing X_t in or make this function get this information for all features in X.
Another thing to note is that I think the DFSTransformer doesn't set the feature provenance, so that might make things tricky. Maybe we need to have DFSTransformer set the feature provenance.
Contributor guide
Assessment
This issue has not been assessed yet.