citusdata / citusdata/citus

Count distinct approximation returns NULL when shards are pruned away

Open
#822 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

To reproduce:

```
CREATE TABLE test (x int, y int);
SELECT master_create_distributed_table('test', 'x', 'append');
SELECT master_create_empty_shard('test');
SET citus.count_distinct_error_rate TO 0.1;
SELECT count(distinct y) FROM test WHERE false;
count
-------

(1 row)
```

Note that the above example works correctly for hash-distributed tables, since it is handled by the router planner (and not approximated).

The issue is that internally we translate the count(distinct ..) call to an hll_cardinality(..) call that returns NULL in case of 0 rows, unlike count(distinct ..) which should return 0.

This issue is related to #93, which is fixed, but it needs to be fixed separately for approximate distinct counts.

Contributor guide

Open the contributing guide

Research direction

Start with the SQL reproduction in the issue and trace the approximate count(distinct y) path that translates the aggregate to hll_cardinality when shards are pruned away. Confirm the empty-result behavior, then verify that the same query returns 0 rather than NULL while preserving the existing hash-distributed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql, sql
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.