DBSCAN/HDSBCAN modify distance matrix in-place
- Dominant language
- Python
- Stars
- 11.7k
- Forks
- 920
- PR merge metrics
- No merged PRs in 30d
Description
Both DBSCAN and HDBSCAN modify the distance matrix passed to the `fit` method. This modification affects the [underperforming group](https://github.com/cleanlab/cleanlab/pull/838) issue manager (under development at the time of creating this issue).
* Modification of the distance matrix can cause unexpected results from Datalab. Since the computed KNN graph is stored as part of `info`, any issue managers that reuse the knn-graph generated by the underperforming group issue manager might not function as expected. Further, computation of `nearest_neighbor` and `distance_to_nearest_neighbor` as part of the issue manager's info can also produce incorrect output.
* A temporary solution for this problem involves passing a copy of the distance matrix to the `fit` method. However, creating a copy of a large distance matrix can be memory intensive for users.
This behaviour is being tracked in Scikit-Learn's repository [[27508](https://github.com/scikit-learn/scikit-learn/issues/27508)]. However, the proposed solution for Scikit-Learn's future releases is to internally create a copy of the distance matrix, which will not fix the memory issue.
Contributor guide
Assessment
This issue has not been assessed yet.