pingcap / pingcap/tiflash

The result of scan context is wrong when query at the first time

Open
#7,848 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component/storage severity/minor type/bug
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)
$ tiup playground nightly

$ tiup bench tpch prepare --host 127.0.0.1 --port 4000

$ mysql --comments --host 127.0.0.1 --port 4000 -u root
mysql> alter database test set tiflash replica 1;
Query OK, 0 rows affected (0.66 sec)
In total 8 tables: 8 succeed, 0 failed, 0 skipped

mysql> use test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from information_schema.tiflash_replica;
+--------------+------------+----------+---------------+-----------------+-----------+----------+
| TABLE_SCHEMA | TABLE_NAME | TABLE_ID | REPLICA_COUNT | LOCATION_LABELS | AVAILABLE | PROGRESS |
+--------------+------------+----------+---------------+-----------------+-----------+----------+
| test         | region     |       98 |             1 |                 |         1 |        1 |
| test         | nation     |       96 |             1 |                 |         1 |        1 |
| test         | part       |      100 |             1 |                 |         1 |        1 |
| test         | supplier   |      102 |             1 |                 |         1 |        1 |
| test         | partsupp   |      104 |             1 |                 |         1 |        1 |
| test         | lineitem   |      110 |             1 |                 |         1 |        1 |
| test         | customer   |      106 |             1 |                 |         1 |        1 |
| test         | orders     |      108 |             1 |                 |         1 |        1 |
+--------------+------------+----------+---------------+-----------------+-----------+----------+
8 rows in set (0.00 sec)

mysql> explain analyze select count(*) from orders where O_CUSTKEY in (0, 150000);
+------------------------------+------------+---------+--------------+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------+-----------+------+
| id                           | estRows    | actRows | task         | access object | execution info                                                                                                                                                                                                                                                                                                                  | operator info                              | memory    | disk |
+------------------------------+------------+---------+--------------+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------+-----------+------+
| StreamAgg_12                 | 1.00       | 1       | root         |               | time:62.5ms, loops:2, RU:0.000000                                                                                                                                                                                                                                                                                               | funcs:count(1)->Column#10                  | 388 Bytes | N/A  |
| └─TableReader_41             | 25.35      | 0       | root         |               | time:62.5ms, loops:1, cop_task: {num: 1, max: 0s, proc_keys: 0, copr_cache_hit_ratio: 0.00}                                                                                                                                                                                                                                     | MppVersion: 2, data:ExchangeSender_40      | 301 Bytes | N/A  |
|   └─ExchangeSender_40        | 25.35      | 0       | mpp[tiflash] |               | tiflash_task:{time:61.5ms, loops:0, threads:3}                                                                                                                                                                                                                                                                                  | ExchangeType: PassThrough                  | N/A       | N/A  |
|     └─Selection_39           | 25.35      | 0       | mpp[tiflash] |               | tiflash_task:{time:61.5ms, loops:0, threads:3}                                                                                                                                                                                                                                                                                  | in(test.orders.o_custkey, 0, 150000)       | N/A       | N/A  |
|       └─TableFullScan_38     | 1262431.00 | 583494  | mpp[tiflash] | table:orders  | tiflash_task:{time:61.5ms, loops:10, threads:3}, tiflash_scan:{dtfile:{total_scanned_packs:144, total_skipped_packs:381, total_scanned_rows:1179648, total_skipped_rows:3105483, total_rs_index_load_time: 0ms, total_read_time: 18ms}, total_create_snapshot_time: 0ms, total_local_region_num: 1, total_remote_region_num: 1} | pushed down filter:empty, keep order:false | N/A       | N/A  |
+------------------------------+------------+---------+--------------+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------+-----------+------+
5 rows in set (0.07 sec)

mysql> explain analyze select count(*) from orders where O_CUSTKEY in (0, 150000);
+------------------------------+------------+---------+--------------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------+-----------+------+
| id                           | estRows    | actRows | task         | access object | execution info                                                                                                                                                                                                                                                                                                          | operator info                              | memory    | disk |
+------------------------------+------------+---------+--------------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------+-----------+------+
| StreamAgg_12                 | 1.00       | 1       | root         |               | time:90.1ms, loops:2, RU:0.000000                                                                                                                                                                                                                                                                                       | funcs:count(1)->Column#10                  | 388 Bytes | N/A  |
| └─TableReader_41             | 25.35      | 0       | root         |               | time:90ms, loops:1, cop_task: {num: 1, max: 0s, proc_keys: 0, copr_cache_hit_ratio: 0.00}                                                                                                                                                                                                                               | MppVersion: 2, data:ExchangeSender_40      | 298 Bytes | N/A  |
|   └─ExchangeSender_40        | 25.35      | 0       | mpp[tiflash] |               | tiflash_task:{time:87.2ms, loops:0, threads:1}                                                                                                                                                                                                                                                                          | ExchangeType: PassThrough                  | N/A       | N/A  |
|     └─Selection_39           | 25.35      | 0       | mpp[tiflash] |               | tiflash_task:{time:87.2ms, loops:0, threads:1}                                                                                                                                                                                                                                                                          | in(test.orders.o_custkey, 0, 150000)       | N/A       | N/A  |
|       └─TableFullScan_38     | 1262431.00 | 1500000 | mpp[tiflash] | table:orders  | tiflash_task:{time:78.2ms, loops:50, threads:1}, tiflash_scan:{dtfile:{total_scanned_packs:350, total_skipped_packs:0, total_scanned_rows:2856754, total_skipped_rows:0, total_rs_index_load_time: 0ms, total_read_time: 39ms}, total_create_snapshot_time: 0ms, total_local_region_num: 3, total_remote_region_num: 0} | pushed down filter:empty, keep order:false | N/A       | N/A  |
+------------------------------+------------+---------+--------------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------+-----------+------+
5 rows in set (0.09 sec)
2. What did you expect to see? (Required)

total_skipped_packs:0 and the scan context keep the same all the time:

tiflash_task:{time:78.2ms, loops:50, threads:1}, tiflash_scan:{dtfile:{total_scanned_packs:350, total_skipped_packs:0, total_scanned_rows:2856754, total_skipped_rows:0, total_rs_index_load_time: 0ms, total_read_time: 39ms}, total_create_snapshot_time: 0ms, total_local_region_num: 3, total_remote_region_num: 0}
3. What did you see instead (Required)
tiflash_task:{time:61.5ms, loops:10, threads:3}, tiflash_scan:{dtfile:{total_scanned_packs:144, total_skipped_packs:381, total_scanned_rows:1179648, total_skipped_rows:3105483, total_rs_index_load_time: 0ms, total_read_time: 18ms}, total_create_snapshot_time: 0ms, total_local_region_num: 1, total_remote_region_num: 1}
4. What is your TiFlash version? (Required)

master

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with tiup playground, the TPC-H preparation steps, and two identical EXPLAIN ANALYZE queries against TiFlash. Compare the tiflash_task and tiflash_scan context between the first and second executions; done means repeated queries retain the expected scan context and pack statistics.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.