pingcap / pingcap/tiflash

TiFlash can not refresh schema automatically after loss column modification

Open
#10,248 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

affects-7.5 affects-8.1 affects-8.5 component/storage severity/moderate type/bug
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)
# table supplier should be large. (maybe more than 100 MiB)
# It is to make sure TiFlash can get the temporary schema in TiDB.
mysql> show create table supplier;
+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table    | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                         |
+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| supplier | CREATE TABLE `supplier` (
  `S_SUPPKEY` bigint NOT NULL,
  `S_NAME` char(25) NOT NULL,
  `S_ADDRESS` varchar(40) NOT NULL,
  `S_NATIONKEY` bigint NOT NULL,
  `S_PHONE` char(15) NOT NULL,
  `S_ACCTBAL` decimal(15,2) NOT NULL,
  `S_COMMENT` varchar(101) NOT NULL,
  `tcol` varchar(256) DEFAULT NULL,
  PRIMARY KEY (`S_SUPPKEY`) /*T![clustered_index] CLUSTERED */
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)

mysql> alter table supplier set tiflash replica 1;
Query OK, 0 rows affected (0.04 sec)

mysql> alter table supplier modify column tcol varchar(255) default null;
Query OK, 0 rows affected (19.33 sec)

TiFlash schema:

$ cat t_115.sql
ATTACH TABLE t_115
(
    s_suppkey Int64, 
    s_name StringV2, 
    s_address StringV2, 
    s_nationkey Int64, 
    s_phone StringV2, 
    s_acctbal Decimal(15, 2), 
    s_comment StringV2, 
    tcol Nullable(StringV2), 
    `_col$_tcol_0` Nullable(StringV2)
)
ENGINE = DeltaMerge(s_suppkey, '{"cols":[{"id":1,"name":{"L":"s_suppkey","O":"s_suppkey"},"offset":0,"state":5,"type":{"Charset":"binary","Collate":"binary","Decimal":0,"Flag":4099,"Flen":20,"Tp":8}},{"id":2,"name":{"L":"s_name","O":"s_name"},"offset":1,"state":5,"type":{"Charset":"utf8mb4","Collate":"utf8mb4_bin","Decimal":0,"Flag":4097,"Flen":25,"Tp":254}},{"id":3,"name":{"L":"s_address","O":"s_address"},"offset":2,"state":5,"type":{"Charset":"utf8mb4","Collate":"utf8mb4_bin","Decimal":0,"Flag":4097,"Flen":40,"Tp":15}},{"id":4,"name":{"L":"s_nationkey","O":"s_nationkey"},"offset":3,"state":5,"type":{"Charset":"binary","Collate":"binary","Decimal":0,"Flag":4097,"Flen":20,"Tp":8}},{"id":5,"name":{"L":"s_phone","O":"s_phone"},"offset":4,"state":5,"type":{"Charset":"utf8mb4","Collate":"utf8mb4_bin","Decimal":0,"Flag":4097,"Flen":15,"Tp":254}},{"id":6,"name":{"L":"s_acctbal","O":"s_acctbal"},"offset":5,"state":5,"type":{"Charset":"binary","Collate":"binary","Decimal":2,"Flag":4097,"Flen":15,"Tp":246}},{"id":7,"name":{"L":"s_comment","O":"s_comment"},"offset":6,"state":5,"type":{"Charset":"utf8mb4","Collate":"utf8mb4_bin","Decimal":0,"Flag":4097,"Flen":101,"Tp":15}},{"id":8,"name":{"L":"tcol","O":"tcol"},"offset":7,"state":5,"type":{"Charset":"utf8mb4","Collate":"utf8mb4_bin","Decimal":0,"Flag":0,"Flen":256,"Tp":15}},{"id":9,"name":{"L":"_col$_tcol_0","O":"_col$_tcol_0"},"offset":8,"state":3,"type":{"Charset":"utf8mb4","Collate":"utf8mb4_bin","Decimal":0,"Flag":0,"Flen":255,"Tp":15}}],"id":115,"index_info":[],"is_common_handle":false,"keyspace_id":4294967295,"name":{"L":"supplier","O":"supplier"},"pk_is_handle":true,"state":5,"tiflash_replica":{"Available":true,"Count":1},"update_timestamp":458808485532663817}', 0)

If there is no DDL trigger TiFlash sync table schema, tcol will not be drop and _col$_tcol_0 will not be renamed.

4. What is your TiFlash version? (Required)

v7.5 ~ master

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the t_115.sql schema and reproduce the supplier table workflow, including the TiFlash replica and ALTER TABLE MODIFY COLUMN commands. Trace the DDL trigger and TiFlash table-schema synchronization path described in the report. Done means the old tcol column is dropped and _col$_tcol_0 is renamed after the column modification without manual intervention.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.