apache / apache/datasketches-java

Compress TDigest can generate centroid with NaN mean

Open
#702 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
958
Forks
226
Avg merge
3d 9h
Merged PRs (30d)
8

Description

See this thread https://github.com/apache/datasketches-rust/pull/23#discussion_r2622754228

Also we don't verify that the centroid mean is not NaN and the weight is not zero (should be positive even?) on deserialization. (Generally we can't assume the data to be deserialized is safe.)

I think the rank/quantile method can return NaN if the calculation result in that. But the stored centroid should not be NaN or else stable sort is unsound?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked pull-request discussion and trace the TDigest compression and deserialization paths mentioned in the issue. Verify how centroid means and weights are validated, and confirm that stored centroids cannot contain NaN means or non-positive weights while rank/quantile behavior remains addressed separately.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.