dask / dask/dask-ml

Incremental wrapper fails for IncrementalPCA

Open
#917 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
951
Forks
262
PR merge metrics
No merged PRs in 30d

Description

**What happened**:
When calling the `fit()` function on the Incremental wrapper with IncrementalPCA, the following error gets thrown:
`AttributeError: 'numpy.ndarray' object has no attribute 'chunks'`. It seems like the Dask Array is internally converted to a Numpy array, which is wrong. I also looked at the scoring parameter, but it is not applicable for PCA and should not cause any issues during fit.

**What you expected to happen**:
The Incremental wrapper should not convert the dask array to a Numpy array internally.

**Minimal Complete Verifiable Example**:

```from dask_ml.datasets import make_classification
from dask_ml.decomposition import IncrementalPCA
from dask_ml.wrappers import Incremental

X, _ = make_classification(n_samples=100000, n_features=100, chunks=10000)
pca = IncrementalPCA(n_components=8, batch_size=40000)
inc = Incremental(pca)
inc.partial_fit(X)
pca.partial_fit(X) # This works
```

**Environment**:

- Dask version: 2022.1
- Python version: 3.9
- Operating System: Ubuntu
- Install method (conda, pip, source): Conda

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.