dask / dask/dask-ml

Implement MissingIndicator for SimpleImputer.

Open
#494 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
951
Forks
262
PR merge metrics
No merged PRs in 30d

Description

Looks like a `indicator_` attribute was added to `sklearn.impute.SimpleImputer` in https://github.com/scikit-learn/scikit-learn/pull/12583. This is causing the sklearn-dev tests to fail in #221.

Here's the documentation for the new attribute:

```
indicator_ : :class:`sklearn.impute.MissingIndicator`
Indicator used to add binary indicators for missing values.
``None`` if add_indicator is False.
```

Here's the message from the sklearn-dev tests on #221:

```
# test_fit_constant[data1] - tests.test_impute
data = dask.array

@pytest.mark.parametrize("data", [X, dX, df, ddf])
def test_fit_constant(data):
a = sklearn.impute.SimpleImputer(strategy="constant", fill_value=-999.0)
b = dask_ml.impute.SimpleImputer(strategy="constant", fill_value=-999.0)

expected = a.fit_transform(X)
result = b.fit_transform(data)

> assert_estimator_equal(a, b)

tests/test_impute.py:42:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

left = SimpleImputer(add_indicator=False, copy=True, fill_value=-999.0,
missing_values=nan, strategy='constant', verbose=0)
right = SimpleImputer(add_indicator=False, copy=True, fill_value=-999.0,
missing_values=nan, strategy='constant', verbose=0)
exclude = set(), kwargs = {}, left_attrs = ['indicator_', 'statistics_']
right_attrs = ['statistics_']

def assert_estimator_equal(left, right, exclude=None, **kwargs):
"""Check that two Estimators are equal

Parameters
----------
left, right : Estimators
exclude : str or sequence of str
attributes to skip in the check
kwargs : dict
Passed through to the dask `assert_eq` method.

"""
left_attrs = [x for x in dir(left) if x.endswith("_") and not x.startswith("_")]
right_attrs = [x for x in dir(right) if x.endswith("_") and not x.startswith("_")]
if exclude is None:
exclude = set()
elif isinstance(exclude, str):
exclude = {exclude}
else:
exclude = set(exclude)

> assert (set(left_attrs) - exclude) == set(right_attrs) - exclude
E AssertionError

dask_ml/utils.py:81: AssertionError
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.