scikit-learn / scikit-learn/scikit-learn
BallTree query match time is O(n) not O(log(n))
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 67.3k
- Forks
- 27.4k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 58
Description
I've run performance analysis on matching NN with BallTree (same with KDTree), and the matching time is linear to number of elements, and should be O(log(n)).
Here are the result of benchmark:
num_elements, match_time
10000 0.09097146987915039
20000 0.18194293975830078
40000 0.3668830394744873
80000 0.7527577877044678
Here is my code:
from sklearn.neighbors import BallTree
import numpy as np
import time
def tree_perf(tree_size):
X = np.random.rand(tree_size, 512)
Y1 = np.random.rand(1, 512)
Y2 = np.random.rand(10, 512)
ts = time.time()
kdt = BallTree(X, leaf_size=30, metric='euclidean')
load_tree = time.time() - ts
num_nn = 1
ts = time.time()
vs = kdt.query(Y1, k=num_nn, return_distance=True)
match1 = time.time() - ts
ts = time.time()
vs = kdt.query(Y2, k=num_nn, return_distance=True)
match10 = time.time() - ts
print(tree_size, load_tree, match1, match10)
print("num_elements", "load_tree", "match_1", "match_10")
for i in range(100):
tree_size = 10000 + i * 10000
tree_perf(tree_size)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the supplied BallTree benchmark, including the 512-dimensional Euclidean data, and compare it with KDTree behavior. Trace the BallTree and KDTree query entry points to determine whether the observed scaling is expected for this workload or indicates a regression. Done means the cause is established and, if a defect is confirmed, a focused fix and benchmark or regression test demonstrate the intended behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100