Simple query containing CTE skips pre-aggs and is pushed down to data source
- Dominant language
- Rust
- Stars
- 20.8k
- Forks
- 2.1k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 181
Description
**Describe the bug**
Cube query planner incorrectly pushes down query containing CTEs to the data source instead of serving it with preaggregation and post processing when query **pushdown** is **enabled**. Release v1.6.48 did not push down the query and served it from pre-aggregation, but ever since, barring releases v1.6.49 to somewhere before v1.7.10 which had a different bug, the query is being pushed down to the datasource.
With the current release 1.7.33, when pushdown is disabled, the query is serviced from the pre-aggregations. When pushdown is enabled, it is pushed down to the data source.
**To Reproduce**
Steps to reproduce the behavior:
1) Define 2 cubes with pre-aggregations where the pre-aggregations' results can be join in a SQL query.
2) Disable query pushdown. Execute a query which joins the result of the pre-aggregation and obeserve the query being serviced from the cube store/pre-aggregations.
3) Enable query pushdown, execute the query again and view the query being pushed down.
**Expected behavior**
If a query can be serviced from pre-aggregation/cubestore it should, regardless if pushdown is enabled or not.
**Screenshots**
Using cubecloud v1.7.33 with pushdown disabled query is serviced from pre-aggregations:
With pushdown enabled query is not serviced from pre-aggregations:
**Minimally reproducible Cube Schema**
This should be easy to test with most cube projects, containing 2 cubes with pre-aggregation and some join column between the 2 pre-aggregation results.
**Version:**
Behaviour changed since v1.6.49 and is present in latest version v1.7.33
(was working as expected in v1.6.48)
**Additional Comments:**
- Tested v1.6.48 using docker image locally to confirm behaviour changed.
- Data source - redshift
Contributor guide
Research direction
No source file or test is named. Start by reproducing the two-cube pre-aggregation query with pushdown disabled and enabled, then trace query-planner handling of CTEs, pushdown, and pre-aggregation selection. Done means a query eligible for Cube Store pre-aggregations uses them regardless of pushdown being enabled, with regression coverage for both settings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100