Wrong pre aggregation result for boolean dimension containing NULL values
- Dominant language
- Rust
- Stars
- 20.8k
- Forks
- 2.1k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 181
Description
**Describe the bug**
Hi, we noticed that some of our preAggregations are returning inconsistent values for boolean dimensions. At first, we believed that this might be a import issue and cubestore was simply reporting the data it had, but we queried it directly and confirmed that something strange is happening. With a direct connection to cubestore on mysql port we did the following:
```sql
SELECT
my_model__bool_col,
COUNT(*) AS total_entries
FROM prod_pre_aggregations.my_model_pre_agg GROUP BY 1;
```
The query returned
```
+--------------------+---------------+
| my_model__bool_col | total_entries |
+--------------------+---------------+
| true | 9393 |
| false | 3888043 |
+--------------------+---------------+
```
`true` and `false` values are reported correctly but it's missing the NULLs. Strangely, we can do the following
```sql
SELECT COUNT(*) AS total_entries
FROM prod_pre_aggregations.my_model_pre_agg
WHERE my_model__bool_col IS NULL;
```
And it reports the correct NULL values
```
+---------------+
| total_entries |
+---------------+
| 464018 |
+---------------+
```
We're using a deployment with separate workers and a router, but couldn't find anything wrong with our configuration. Besides that, every query that's not aggregating on a boolean dimension seems to work fine, so it seems unlikely that this is caused by a miss configuration.
I even patched cube to log the partial data on each worker before it is returned to the main one and individually each worker already responded with a wrong aggregation (missing the nulls, even tough the partitions contained it)
**To Reproduce**
I didn't manage to reproduce this with artificial data, but I'll update the issue as soon as I have. I'm opening it now in case somebody has experienced the same thing
**Version:**
0.34.0
Contributor guide
Assessment
This issue has not been assessed yet.