Something not right with serialisation of python (sk-learn) models
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 163
- Forks
- 46
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 5
Description
using MLJ
model = @load RandomForestClassifier pkg=ScikitLearn
data = load_iris()
y, X = unpack(data, ==(:target), n->true, rng=1234)
mach = machine(model, X, y) |> fit!
MLJ.save("junk.jlso", mach)
## NEW REPL SESSION
using MLJ
model = @load RandomForestClassifier pkg=ScikitLearn
mach_predict_only = machine("junk.jlso")
Sometimes the last expression hangs and if I interrupt Julia crashes. Sometimes it works fine. Same is true for JLSO.load("junk.jlso"). Anyone able to recreate?
(junk) pkg> st
Status `~/Dropbox/Julia7/MLJ/MLJ/sandbox/junk/Project.toml`
[9da8a3cd] JLSO v2.3.2
[add582a8] MLJ v0.14.1
[5ae90465] MLJScikitLearnInterface v0.1.6
julia> versioninfo()
Julia Version 1.5.1
Commit 697e782ab8 (2020-08-25 20:08 UTC)
Platform Info:
OS: macOS (x86_64-apple-darwin19.5.0)
CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
JULIA_LTS_PATH = /Applications/Julia-1.0.app/Contents/Resources/julia/bin/julia
JULIA_PATH = /Applications/Julia-1.5.app/Contents/Resources/julia/bin/julia
JULIA_NUM_THREADS = 5
It may be that the python objects bound to fitresult are not reliably persistent and we need to do implement custom MMI.save and MMI.restore along the lines of XGBoost.
Assuming ScikitLearn.jl exposes the python serialization, this ought to be easy to do.
@OkonSamuel Thoughts?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the provided MLJ.save and machine("junk.jlso") sequence in a fresh Julia REPL using the listed Julia, MLJ, JLSO, and ScikitLearnInterface versions. Inspect the ScikitLearn interface's serialization behavior and compare it with the MMI.save and MMI.restore implementation linked for XGBoost. Done means loading the saved model completes reliably without hanging or crashing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia, python, scikit-learn
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100