alteryx / alteryx/evalml

Add ability to see if a base feature will have an impact on predictions

Open
#3,818 0 comments 0 reactions 0 assignees View on GitHub
new feature
Dominant language
Python
Stars
850
Forks
96
PR merge metrics
No merged PRs in 30d

Description

- As a user, I wish I could easily see whether a feature in X will be passed to the estimator in a pipeline.

While implementing #3753, it became clear that there are times when you can look at a fit pipeline and know whether or not a feature in X will have an impact on predictions by determining if the feature is ultimately passed to the estimator. If a feature isn't in the transformed dataset produced by `pipeline.transform_all_but_final`, then it's not passed to the estimator and therefore has no impact on predictions! Such features should have a feature importance of 0 and a partial dependence that is the same for all feature values. Those can be lengthy calculations, but we already have the information that those measures would tell us, so we shouldn't need to make those calculations.

If we had an easy way to see this information, it could be leveraged to avoid unnecessary computation!

Situations where this may happen:
- RF Regressor/Classifier Feature Selector determines that a feature will be unimportant and drops it
- `DropColumns` transformer is used to drop specific columns
- `SelectColumns` transformer is used and leaves some columns out

#### Code Example

A potential user-facing api:
```python
pipeline.is_feature_impactful(feature_name)
```

The implementation
```python
def is_feature_impactful(self, X, feature_name)
X_t = self.transform_all_but_final(X)

feature_provenance = self._get_feature_provenance()
return feature_name in feature_provenance or feature name in X_t.columns
```

Note that doing the `pipeline.transform_all_but_final(X)` calculation over and over for every feature is unnecessary repeted computation. So we could either allow passing X_t in or make this function get this information for all features in X.

Another thing to note is that I think the DFSTransformer doesn't set the feature provenance, so that might make things tricky. Maybe we need to have DFSTransformer set the feature provenance.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.