scikit-learn / scikit-learn/scikit-learn

Suggestion: Fill with feature mean instead of 0 for VarianceThreshold.inverse_transform

Open
#14,094 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Enhancement module:feature_selection Needs Decision
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

Today, sklearn.feature_selection.VarianceThreshold.inverse_transform method fills in zeros to features which were removed for having too-small variance.
This is certainly predictable, easy to implement and easy to explain.

However, filling in zeros without respect to the data passed to fit means that the reconstruction error can become arbitrarily large. For example, suppose that one of the features in your data always takes the value 10**6. This clearly has zero variance, since it always takes the same value; however, filling in zeros for that feature when the data via transform and inverse_transform will result in an output which dramatically differs from the input.

Instead, I think it would make sense to fill in the mean of the columns when using inverse_transform. The means can be computed and stored from the data passed to fit. This will make the reconstruction of the transformed data via inverse_transform more closely reflect the data that was passed to fit because any columns which are removed for having variance less than threshold must, by definition, be tightly grouped about the sample mean.

Naturally, in the special cases where the sample means of input features passed to fit are already zero, the proposed inverse_transform method will function the same way it does today, as it will fill in zero values for those features.

In terms of code, this just means keeping an array of column means in addition to the indices of the removed columns.

Of course, it's possible that I have missed an important subtlety, or there is a competing concern which outweighs the argument that I've outlined here. If that's the case, I'd like to know what I've missed!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading VarianceThreshold.inverse_transform together with how fit stores information and how transform/inverse_transform reconstruct features. Compare the current zero-filling behavior with the proposed fitted column means, then determine the compatibility and API implications; done means the behavior is resolved and the reconstruction is verified against the stated constant-feature example.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.