SQL API: option to reject queries that don't fully push down (instead of silently falling back to in-memory)
- Dominant language
- Rust
- Stars
- 20.8k
- Forks
- 2.1k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 181
Description
**Is your feature request related to a problem? Please describe.**
On the SQL API (/v1/cubesql), a query that can't be fully pushed down to the data source silently falls back to Cube/DataFusion in-memory execution. Cube fetches up to the row cap (CUBEJS_DB_QUERY_LIMIT, default 50k) as an intermediate result, post-processes it in memory, then applies the user's LIMIT. The returned rows are computed over a truncated, arbitrarily-ordered subset & silently incorrect, with no error signal. For analytics/agent use cases where correctness matters, this is a footgun: users (and LLM agents) can't tell a wrong answer from a right one.
There's currently no way to (a) make Cube reject such queries, or (b) reliably detect on the execution path whether a query fully pushed down. CUBESQL_SQL_PUSH_DOWN only toggles whether pushdown is attempted, and the JS config hooks (checkSqlAuth, queryRewrite) don't see the pushdown decision (it's made in the Rust cubesql layer).
**Describe the solution you'd like**
Either or both:
1. Enforcement mode: a config flag (e.g. CUBESQL_REQUIRE_PUSHDOWN / per-request option) that makes the SQL API return an error instead of executing in memory when a query doesn't fully push down.
2. A first-class pushdown signal on the execution path e.g. a response field/header, or a documented, stable way to obtain "did this fully push down?" for a given SQL query (analogous to the sql4sql status in #10591, but on /v1/cubesql).
**Describe alternatives you've considered**
We built a transparent reverse proxy in front of /v1/cubesql that, before executing, runs EXPLAIN and inspects the physical-plan root node: a fully pushed-down query roots at CubeScanExecutionPlan (the scan reports a single output partition, so no plumbing wrapper is added); any other root means an operator runs above the scan → block. This works but: it adds an EXPLAIN round-trip per query, relies on parsing plan text (version-fragile), and every consumer must front Cube with it rather than Cube enforcing centrally. A built-in option would be far more robust.
**Additional context**
- Affects correctness governance when multiple services share one Cube SQL API.
- The fallback is invisible today; even a clear error would be a major improvement over silent truncation
We would be happy to work on the contribution if you are open to PRs
Contributor guide
Research direction
The request targets the /v1/cubesql Rust cubesql execution path; start by tracing where the pushdown decision is made and how CUBESQL_SQL_PUSH_DOWN is applied. Compare the EXPLAIN physical-plan inspection described in the issue, and define completion as either explicit rejection of non-fully-pushed-down queries or a stable per-query pushdown signal. No specific files or tests are named.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100