scikit-learn / scikit-learn/scikit-learn

RFC/API (Array API) mixing devices and data types with estimators

Open
#26,083 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

API Array API RFC
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

Right now, if the user fits an estimator using a pandas.DataFrame, but passes a numpy.ndarray during predict, they get a warning due to missing feature names.

The situation is only to get more complicated as we're adding support for more types via array API.

Some related issues here are:

  • device: data during fit sits on a GPU, but a CPU is used for predict (with the same data type)
  • types: using one type to fit, and use another type during predict: how do we handle this both in terms of device and the type? Do we let the operator figure out if they can coerce the data into the type which can be used?
  • persistence: how do we let users fit on one device, but load on another device
  • estimator conversion: do we let users convert an estimator which is fit using one type/device, to an estimator compatible with another type/device?

I vaguely remember us talking about some of these issues, but I don't see any active discussion. I might have missed something.

Related: in a library like pytorch, you can decide which device is going to be used when you load a model's weights.

cc @thomasjpfan

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the estimator fit and predict API behavior described in the issue, along with the related device, type, persistence, and estimator-conversion discussions. This is an RFC rather than an implementation task; done requires an agreed policy for mixing data types and devices before specific code changes can be identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
api, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.