Handling of missings
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 163
- Forks
- 46
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 5
Description
Not sure if it's the appropriate place to post this; it looks like we could improve the situations when dealing with missings. Here's a simple example of something that's not ideal:
yhas some missingsXdoesn't- we drop the rows of
yandXthat have missings - we feed that in a machine
- the machine fails because the model doesn't handle
Union{Missing,Float64}
of course there are no missings in y anymore. Interestingly, the scientific type is correct so that could be exploited but the machine type has Union{Missing,...}
Maybe we could help with:
- provide a
dropmissingfunction which "re-coerces" - have a mechanism in
fitwhich checks whether the type and scientific type match in terms of missing, if they don't and ST has it right, maybe the data can be re-coerced
thoughts?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing MLJBase's existing handling of missing values, scientific types, and the fit path described in the issue. The scope is not settled: compare the proposed dropmissing re-coercion with fit-time checking, then clarify the intended behavior and add tests demonstrating that cleaned data no longer retains an incompatible missing-value type.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100