pingcap / pingcap/tiflash

TIMESTAMPDIFF(YEAR, dbirth, CURDATE()) is very slow if dbirth is of decimal(10,0)

Open
#5,074 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/performance
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)
create table t (id int, dbirth decimal(10,0), birth date);
alter table t set tiflash replica 1;
insert into t values (0, 20220501,'20120501'),(3, 20210503,'20220503'),(4, 20220504,'20220504'),(5, 20220505,'20220505'),(200214, 20020514,'20020514'),(200114, 20010514,'20010514'),(201114, 20110514,'20110514');
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
insert into t (id, dbirth,birth) select * from t;
2. What did you expect to see? (Required)

the function does not cost too much time compared to executing birthday of data type.

3. What did you see instead (Required)

evaluating on birthday of decimal is very slower than on birthday of date.

mysql> explain analyze SELECT  max(TIMESTAMPDIFF(YEAR, dbirth, CURDATE())) from t group by id limit 10;
+----------------------------------------+-----------+---------+--------------+---------------+--------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+------+
| id                                     | estRows   | actRows | task         | access object | execution info                                                                             | operator info                                                                               | memory | disk |
+----------------------------------------+-----------+---------+--------------+---------------+--------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+------+
| Limit_10                               | 10.00     | 10      | root         |               | time:9.85s, loops:2                                                                        | offset:0, count:10                                                                          | N/A    | N/A  |
| └─TableReader_38                       | 10.00     | 17      | root         |               | time:9.85s, loops:1, cop_task: {num: 1, max: 0s, proc_keys: 0, copr_cache_hit_ratio: 0.00} | data:ExchangeSender_37                                                                      | N/A    | N/A  |
|   └─ExchangeSender_37                  | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:9.85s, loops:1, threads:8}                                              | ExchangeType: PassThrough                                                                   | N/A    | N/A  |
|     └─Projection_33                    | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:9.85s, loops:1, threads:8}                                              | Column#5                                                                                    | N/A    | N/A  |
|       └─HashAgg_34                     | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:9.85s, loops:1, threads:1}                                              | group by:test.t.id, funcs:max(Column#7)->Column#5                                           | N/A    | N/A  |
|         └─ExchangeReceiver_36          | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:9.85s, loops:8, threads:8}                                              |                                                                                             | N/A    | N/A  |
|           └─ExchangeSender_35          | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:9.85s, loops:256, threads:6}                                            | ExchangeType: HashPartition, Hash Cols: [name: test.t.id, collate: binary]                  | N/A    | N/A  |
|             └─HashAgg_16               | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:9.85s, loops:256, threads:1}                                            | group by:Column#11, funcs:max(Column#10)->Column#7                                          | N/A    | N/A  |
|               └─Projection_52          | 983040.00 | 3932172 | mpp[tiflash] |               | tiflash_task:{time:9.83s, loops:64, threads:6}                                             | timestampdiff(YEAR, cast(test.t.dbirth, datetime BINARY), 2022-06-06)->Column#10, test.t.id | N/A    | N/A  |
|                 └─TableFullScan_32     | 983040.00 | 3932172 | mpp[tiflash] | table:t       | tiflash_task:{time:9.36s, loops:64, threads:6}                                             | keep order:false                                                                            | N/A    | N/A  |
+----------------------------------------+-----------+---------+--------------+---------------+--------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+------+
10 rows in set (9.85 sec)

mysql> explain analyze SELECT  max(TIMESTAMPDIFF(YEAR, birth, CURDATE())) from t group by id limit 10;
+----------------------------------------+-----------+---------+--------------+---------------+---------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+--------+------+
| id                                     | estRows   | actRows | task         | access object | execution info                                                                              | operator info                                                              | memory | disk |
+----------------------------------------+-----------+---------+--------------+---------------+---------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+--------+------+
| Limit_10                               | 10.00     | 10      | root         |               | time:92.1ms, loops:2                                                                        | offset:0, count:10                                                         | N/A    | N/A  |
| └─TableReader_38                       | 10.00     | 17      | root         |               | time:92.1ms, loops:1, cop_task: {num: 1, max: 0s, proc_keys: 0, copr_cache_hit_ratio: 0.00} | data:ExchangeSender_37                                                     | N/A    | N/A  |
|   └─ExchangeSender_37                  | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:91.2ms, loops:1, threads:8}                                              | ExchangeType: PassThrough                                                  | N/A    | N/A  |
|     └─Projection_33                    | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:91.2ms, loops:1, threads:8}                                              | Column#5                                                                   | N/A    | N/A  |
|       └─HashAgg_34                     | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:91.1ms, loops:1, threads:1}                                              | group by:test.t.id, funcs:max(Column#7)->Column#5                          | N/A    | N/A  |
|         └─ExchangeReceiver_36          | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:90.9ms, loops:7, threads:8}                                              |                                                                            | N/A    | N/A  |
|           └─ExchangeSender_35          | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:90.6ms, loops:256, threads:7}                                            | ExchangeType: HashPartition, Hash Cols: [name: test.t.id, collate: binary] | N/A    | N/A  |
|             └─HashAgg_16               | 10.00     | 17      | mpp[tiflash] |               | tiflash_task:{time:88.4ms, loops:256, threads:1}                                            | group by:Column#11, funcs:max(Column#10)->Column#7                         | N/A    | N/A  |
|               └─Projection_52          | 983040.00 | 3932172 | mpp[tiflash] |               | tiflash_task:{time:63.4ms, loops:64, threads:7}                                             | timestampdiff(YEAR, test.t.birth, 2022-06-06)->Column#10, test.t.id        | N/A    | N/A  |
|                 └─TableFullScan_32     | 983040.00 | 3932172 | mpp[tiflash] | table:t       | tiflash_task:{time:36.9ms, loops:64, threads:7}                                             | keep order:false                                                           | N/A    | N/A  |
+----------------------------------------+-----------+---------+--------------+---------------+---------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+--------+------+
10 rows in set (0.10 sec)
4. What is your TiFlash version? (Required)

master

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided TiFlash reproduction and compare the two EXPLAIN ANALYZE queries, focusing on the decimal-to-datetime path versus the date path. Done means TIMESTAMPDIFF(YEAR, dbirth, CURDATE()) no longer shows the reported severe slowdown relative to the equivalent date column, with regression coverage added where appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.