apache / apache/datasketches-java
can we use stable sort for tdigest
- Lingua principale
- Java
- Stelle
- 958
- Fork
- 226
- Merge medio
- 3g 15h
- PR unite (30g)
- 10
Descrizione
the sort on centroids (v,w) -> (value, weight) in tdigest merge uses the sort based on value (v) and index -> therefore if we have centroids like (200,1)(100,3)(300,2)(100,1) then at merge it can be sorted to 2 states :
1. (100,1)(100,3)(200,1)(300,2)
2. (100,3)(100,1)(200,1)(300,2)
due to which the merging result can differ
other than that we can use sorting based on value (v) and weight (w) as in
https://github.com/tdunning/t-digest/blob/main/core/src/main/java/com/tdunning/math/stats/Sort.java
and this will make the sort stable and remove variance in result
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Direzione di ricerca
Start in the t-digest merge logic where centroids are sorted by value and index. Compare that ordering with core/src/main/java/com/tdunning/math/stats/Sort.java referenced in the issue, then verify that equal-value centroids are ordered deterministically and produce consistent merge results.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- java
- Ambito
- data
- Tipo di issue
- Bug
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Attiva
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 58/100