taking performance issue more seriously
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 163
- Forks
- 46
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 5
Description
I know MLJ will definitely have some overhead since it's wraps other code. But i believe this overhead can be reduced below the current level with careful design considerations. Avoidable overhead which are neglected may come to hunt us when the code is scaled. There are a few things i discovered which are important.
- As pointed out in issue #151
selectrows. This causes overhead when used inevaluatemethod which calls this method a lot of times depending on therepeatsparameter. - In the
MLJBase.fitmethod, thematrixmethod (which to my understanding copies data) is called on a given tableXof course this isn't bad if i call this method only once using the same data changing a couple of model parameters. This becomes important in theevaluatemethod which callsfita couple of times depending onrepeats.(If X is larger this isn't nice). (I don't thinkupdatemethod does enough justice). Also other methods call these methods repeatedly copying X in each case - The return type to
MLJBase.predictfor probabilistic Classifiers.( I can't find the link to the issue)
These are just some of the point. I believe there are other things which affects scalability. It is better we start treating this issues more serious. Imagine a case where One wants to embark on a kaggle competition on a large dataset only to find out that the overhead is just unbearable in this case). You may correct me perhaps i'm missing something.
@ablaom , @tlienart
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by profiling evaluate and its repeated calls to selectrows and MLJBase.fit, then inspect the matrix call on X and the return type of MLJBase.predict for probabilistic classifiers. Review issue #151 and the existing comment discussion to identify an agreed scope. Done should mean a defined set of bottlenecks is addressed and benchmark results show reduced overhead without changing expected behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- machine-learning, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100