scikit-learn / scikit-learn/scikit-learn
RFC/API (Array API) mixing devices and data types with estimators
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 67.3k
- Forks
- 27.4k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 58
Description
Right now, if the user fits an estimator using a pandas.DataFrame, but passes a numpy.ndarray during predict, they get a warning due to missing feature names.
The situation is only to get more complicated as we're adding support for more types via array API.
Some related issues here are:
- device: data during
fitsits on a GPU, but a CPU is used for predict (with the same data type) - types: using one type to
fit, and use another type duringpredict: how do we handle this both in terms of device and the type? Do we let the operator figure out if they can coerce the data into the type which can be used? - persistence: how do we let users fit on one device, but load on another device
- estimator conversion: do we let users convert an estimator which is fit using one type/device, to an estimator compatible with another type/device?
I vaguely remember us talking about some of these issues, but I don't see any active discussion. I might have missed something.
Related: in a library like pytorch, you can decide which device is going to be used when you load a model's weights.
cc @thomasjpfan
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the estimator fit and predict API behavior described in the issue, along with the related device, type, persistence, and estimator-conversion discussions. This is an RFC rather than an implementation task; done requires an agreed policy for mixing data types and devices before specific code changes can be identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, pandas, python
- Domain
- api, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100