apache / apache/datasketches-java
can we use stable sort for tdigest
- Lenguaje dominante
- Java
- Estrellas
- 958
- Forks
- 226
- Merge medio
- 3 d 9 h
- PR fusionados (30 d)
- 8
Descripción
the sort on centroids (v,w) -> (value, weight) in tdigest merge uses the sort based on value (v) and index -> therefore if we have centroids like (200,1)(100,3)(300,2)(100,1) then at merge it can be sorted to 2 states :
1. (100,1)(100,3)(200,1)(300,2)
2. (100,3)(100,1)(200,1)(300,2)
due to which the merging result can differ
other than that we can use sorting based on value (v) and weight (w) as in
https://github.com/tdunning/t-digest/blob/main/core/src/main/java/com/tdunning/math/stats/Sort.java
and this will make the sort stable and remove variance in result
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Línea de trabajo
Start in the t-digest merge logic where centroids are sorted by value and index. Compare that ordering with core/src/main/java/com/tdunning/math/stats/Sort.java referenced in the issue, then verify that equal-value centroids are ordered deterministically and produce consistent merge results.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- java
- Área
- data
- Tipo de issue
- Error
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Estado de actividad
- Activo
- Claridad
- Bastante claro
- Aptitud para principiantes
- 58/100