JuliaAI / JuliaAI/NearestNeighborModels.jl
[testing] check results are coherent vs sklearn when using sample weights
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 27
- Forks
- 1
- Avg merge
- 1d 23m
- Merged PRs (30d)
- 9
Description
We've recently added (https://github.com/alan-turing-institute/MLJModels.jl/pull/125) the possibility to add weights to samples in KNNC, KNNR. It seems fine but it would still be good to check this a bit more and ideally against an external benchmark like Sklearn which I believe supports sample weights as well.
Steps:
be on theedit This now lives at NearestNeighborModels (current repo)devbranch of MLJModels- generate some dummy data with dummy weights (see also examples in tests for NearestNeighbors though it'd be better to use less dumb data where classes overlap a bit)
- save the data and do the same analysis in sklearn
- check that the results look roughly similar (like accuracy within +- 5%)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the examples in the NearestNeighbors tests and review the sample-weight support for KNNC and KNNR. Generate overlapping dummy data with weights, run the same analysis in scikit-learn, and compare the results; done means accuracy is roughly within ±5%.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia, scikit-learn
- Domain
- machine-learning, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100