cube-js / cube-js/cube

Wrong pre aggregation result for boolean dimension containing NULL values

Open
#7,197 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
20.8k
Forks
2.1k
Avg merge
1d 2h
Merged PRs (30d)
181

Description

**Describe the bug**
Hi, we noticed that some of our preAggregations are returning inconsistent values for boolean dimensions. At first, we believed that this might be a import issue and cubestore was simply reporting the data it had, but we queried it directly and confirmed that something strange is happening. With a direct connection to cubestore on mysql port we did the following:

```sql
SELECT
my_model__bool_col,
COUNT(*) AS total_entries
FROM prod_pre_aggregations.my_model_pre_agg GROUP BY 1;
```

The query returned

```
+--------------------+---------------+
| my_model__bool_col | total_entries |
+--------------------+---------------+
| true | 9393 |
| false | 3888043 |
+--------------------+---------------+
```

`true` and `false` values are reported correctly but it's missing the NULLs. Strangely, we can do the following

```sql
SELECT COUNT(*) AS total_entries
FROM prod_pre_aggregations.my_model_pre_agg
WHERE my_model__bool_col IS NULL;
```

And it reports the correct NULL values

```
+---------------+
| total_entries |
+---------------+
| 464018 |
+---------------+

```

We're using a deployment with separate workers and a router, but couldn't find anything wrong with our configuration. Besides that, every query that's not aggregating on a boolean dimension seems to work fine, so it seems unlikely that this is caused by a miss configuration.

I even patched cube to log the partial data on each worker before it is returned to the main one and individually each worker already responded with a wrong aggregation (missing the nulls, even tough the partitions contained it)

**To Reproduce**

I didn't manage to reproduce this with artificial data, but I'll update the issue as soon as I have. I'm opening it now in case somebody has experienced the same thing

**Version:**
0.34.0

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.