pingcap / pingcap/tidb

Make it possible to push Agg down cross UnionScan

Open
#62,975 1 comment 0 reactions 0 assignees View on GitHub
affects-8.5 report/customer sig/planner sig/transaction type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement
```

id | estRows | estCost | actRows | task | access object | execution info | operator info | memory | disk
-- | -- | -- | -- | -- | -- | -- | -- | -- | --
HashAgg_7 | 1.00 | 110461651.48 | 1 | root |   | time:1.97s, loops:2, partial_worker:{wall_time:1.97215596s, concurrency:5, task_num:2561, tot_wait:9.668152304s, tot_exec:183.727117ms, tot_time:9.86055611s, max:1.97211872s, p95:1.97211872s}, final_worker:{wall_time:1.972170191s, concurrency:5, task_num:9, tot_wait:20.22µs, tot_exec:2.73µs, tot_time:9.86074141s, max:1.97215273s, p95:1.97215273s} | funcs:count(1)->Column#24 | 622.0 KB | 0 Bytes
└─UnionScan_9 | 2468991.74 | 61179069.06 | 2580537 | root |   | time:1.96s, loops:2562 | eq(obei_common_platform.t_robot_api_info.arrange_id, "MATC-03-XJPP-01-001") | N/A | N/A
└─IndexReader_11 | 2468991.74 | 61179069.06 | 2580536 | root |   | time:53.5ms, loops:2526, cop_task: {num: 84, max: 74.7ms, min: 241.6µs, avg: 20.8ms, p95: 72.3ms, max_proc_keys: 50144, p95_proc_keys: 50144, tot_proc: 1.67s, tot_wait: 2.45ms, copr_cache_hit_ratio: 0.39, build_task_duration: 21.3µs, max_distsql_concurrency: 4}, rpc_info:{Cop:{num_rpc:84, total_time:1.75s}} | index:IndexRangeScan_10 | 8.38 MB | N/A
└─IndexRangeScan_10 | 2468991.74 | 532561840.92 | 2580536 | cop[tikv] | table:t_robot_api_info, index:idx_arrangeId(ARRANGE_ID) | tikv_task:{proc max:70ms, min:0s, avg: 28ms, p80:50ms, p95:70ms, iters:2853, tasks:84}, scan_detail: {total_process_keys: 1616984, total_process_keys_size: 163315384, total_keys: 1617035, get_snapshot_time: 998.4µs, rocksdb: {delete_skipped_count: 1311, key_skipped_count: 2687466, block: {cache_hit_count: 6078}}}, time_detail: {total_process_time: 1.67s, total_suspend_time: 2.73ms, total_wait_time: 2.45ms, total_kv_read_wall_time: 1.62s, tikv_wall_time: 1.68s} | range:["MATC-03-XJPP-01-001","MATC-03-XJPP-01-001"], keep order:false | N/A | N/A

```

Image

As you can see, when there're dirty writes, the `HashAgg` can not be pushed down to the coprocessor side.
So the computation is left to the TiDB side.
Compared with the normal execution without dirty writes, it's much slower.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.