apache / apache/datasketches-java

can we use stable sort for tdigest

Abierto
#739 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Java
Estrellas
958
Forks
226
Merge medio
3 d 9 h
PR fusionados (30 d)
8

Descripción

the sort on centroids (v,w) -> (value, weight) in tdigest merge uses the sort based on value (v) and index -> therefore if we have centroids like (200,1)(100,3)(300,2)(100,1) then at merge it can be sorted to 2 states :

1. (100,1)(100,3)(200,1)(300,2)
2. (100,3)(100,1)(200,1)(300,2)

due to which the merging result can differ

other than that we can use sorting based on value (v) and weight (w) as in
https://github.com/tdunning/t-digest/blob/main/core/src/main/java/com/tdunning/math/stats/Sort.java

and this will make the sort stable and remove variance in result

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Start in the t-digest merge logic where centroids are sorted by value and index. Compare that ordering with core/src/main/java/com/tdunning/math/stats/Sort.java referenced in the issue, then verify that equal-value centroids are ordered deterministically and produce consistent merge results.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
java
Área
data
Tipo de issue
Error
Dificultad
3/5
Tiempo estimado
1-2 días
Estado de actividad
Activo
Claridad
Bastante claro
Aptitud para principiantes
58/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.