scikit-learn / scikit-learn/scikit-learn
OPTICS Benchmark and improve the performance
Open
Nobody has claimed this yet.
help wanted
module:cluster
Performance
- Dominant language
- Python
- Stars
- 67.3k
- Forks
- 27.4k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 58
Description
Our OPTICS is now O(n^2). We can improve it to O(n * logn) with a heap (See ELKI and R dbscan).
Open this issue for us to benchmark our implementation and (maybe) try to improve it in the future.
Related comments:
- (from kno10) It is worth noting that because of this operation, the current OPTICS in scikit-learn obviously needs n² operations, while using a heap here would reduce this to n*O(log n) in this particular location. At the same time, the n² part has very low constant factors (because of numpy), while attempting to implement an updateable heap efficiently can easily introduce much larger constant factors here...
- (from kno10) https://github.com/scikit-learn/scikit-learn/pull/12421#issuecomment-431708045
ping @espg you can post your results here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named. Start by locating the OPTICS implementation and its benchmark coverage, then measure the current O(n²) operation and compare it with the heap approach described using ELKI and R dbscan. Done means benchmark results are reported and any proposed change demonstrates an improvement without unacceptable constant-factor costs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100