Avoid re-evaluating expressions in filters and projections
- 主要语言
- Rust
- 星标
- 9.3k
- 派生
- 2.4k
- 平均合并
- 3 天 11 小时
- 30 天内合并 PR
- 360
描述
This patterns shows up pretty often:
```sql
select expensive(col)
from t
where expensive(col)
```
A pathological case is variant / json:
```sql
select variant_get(col, 'key')
from t
from variant_get(col, 'key')
```
There's two issues here:
1. Until we solve projection pushdown (https://github.com/apache/datafusion/issues/14993) if `key` is not shredded we materialize the entire `col` and then extract `key` in a `ProjectionExec`.
2. Even once that is resolved, or in the case that `key` is not shredded evaluating `variant_get(col, 'key')` itself is expensive we still re-compute `variant_get(col, 'key')` twice: once for the filter and once for the projection.
贡献指南
调研方向
首先阅读 SQL 示例中描述的过滤和投影行为,包括 ProjectionExec 以及 projection-pushdown issue #14993。确定如何表示和计算重复表达式(例如 variant_get(col, 'key')),然后将完成定义为:在保留查询结果的同时避免重复计算。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- rust, sql
- 领域
- data-engineering, databases
- Issue 类型
- 功能
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100