pingcap / pingcap/tiflash

TiFlash is reporting errors and restarting continuously. [ERROR]: Detected overflow when decoding integer of length 22 with column type UInt64: pk_type is INT64, schema_snapshot->col_id_to_block_pos

Open
#10,681 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component/storage contribution first-time-contributor type/bug
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)
drop table test.tm_test;

CREATE TABLE `test`.`tm_test` (
  `id` bigint NOT NULL,
  `customer_types` json DEFAULT NULL,
  `other_phones` varchar(1024) DEFAULT NULL,
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */,
  KEY `idx_customer_profile_customer_types` ((cast(`customer_types` as unsigned array)))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin COMMENT='测试表';

INSERT INTO test.tm_test (id,customer_types,other_phones) VALUES
	 (40343892505477852,'[1, 0]',NULL),
	 (40343892505477907,'[1]',NULL),
	 (40343892505477915,'[1, 0]',NULL),
	 (40343892505477919,'[1]',NULL),
	 (40564894342709282,'[1, 0]',NULL),
	 (40564894342709318,'[1, 0]',NULL),
	 (40564894342709322,'[1]',NULL),
	 (40564894342709326,'[0]',NULL),
	 (40610284294152202,'[3]',NULL),
	 (40610284294152208,'[0]',''),
	 (40610284294152218,'[1, 0]',NULL),
	 (40610284294152242,'[1]',NULL),
	 (40617455906889734,'[1, 0]',NULL);

ALTER TABLE test.tm_test SET TIFLASH REPLICA 1;
select * from information_schema.tiflash_replica;

ALTER TABLE test.tm_test MODIFY COLUMN other_phones varchar(512) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL;
2. What did you expect to see? (Required)
[2026/01/27 19:31:09.217 +08:00] [ERROR] [RaftCommands.cpp:506] ["[region_id=742705 applied_term=6 applied_index=8] catch exception: Detected overflow when decoding integer of length 35 with column type UInt64: pk_type is INT64, schema_snapshot->col_id_to_block_pos is  [map info : (column_id=-1025, pos=2) (column_id=-1024, pos=1) (column_id=-1, pos=0) (column_id=1, pos=3) (column_id=2, pos=4) (column_id=4, pos=6) (column_id=5, pos=5)  ];, schema_snapshot->col_id_to_def_pos is  [map info : (column_id=-1025, pos=2) (column_id=-1024, pos=1) (column_id=-1, pos=0) (column_id=1, pos=3) (column_id=2, pos=4) (column_id=4, pos=6) (column_id=5, pos=5)  ];,schema_snapshot->column_defines is  [column define : (id=-1, name=_tidb_rowid, type=Int64) (id=-1024, name=_INTERNAL_VERSION, type=UInt64) (id=-1025, name=_INTERNAL_DELMARK, type=UInt8) (id=1, name=id, type=Int64) (id=2, name=customer_types, type=Nullable(String)) (id=5, name=other_phones, type=Nullable(String)) (id=4, name=_v$_idx_customer_profile_customer_types_0, type=Nullable(UInt64))  ];, decoding_snapshot_epoch is 2, block schema is [{\"name\":\"_INTERNAL_DELMARK\",\"id\":-1025,\"type\":\"UInt8\",\"column\":\"UInt8(size = 1)\"},{\"name\":\"_INTERNAL_VERSION\",\"id\":-1024,\"type\":\"UInt64\",\"column\":\"UInt64(size = 1)\"},{\"name\":\"_tidb_rowid\",\"id\":-1,\"type\":\"Int64\",\"column\":\"Int64(size = 0)\"},{\"name\":\"id\",\"id\":1,\"type\":\"Int64\",\"column\":\"Int64(size = 0)\"},{\"name\":\"customer_types\",\"id\":2,\"type\":\"Nullable(String)\",\"column\":\"Nullable(size = 1, String(size = 1), UInt8(size = 1))\"},{\"name\":\"_v$_idx_customer_profile_customer_types_0\",\"id\":4,\"type\":\"Nullable(UInt64)\",\"column\":\"Nullable(size = 0, UInt64(size = 0), UInt8(size = 0))\"},{\"name\":\"other_phones\",\"id\":5,\"type\":\"Nullable(String)\",\"column\":\"Nullable(size = 0, String(size = 0), UInt8(size = 0))\"}] : TiKV value contains: : 00003000200010204030508002B004E00DC42A5758E548F000302000000220000000912000000091A000000010000000000000000000000000000000302000000220000000A120000000A1A00000001000000000000000000000000000000, : 800003000200010204030508001E0034001343A5758E548F00030100000015000000090D00000001000000000000000301000000150000000A0D0000000100000000000000, : 800003000200010204030508002B004E001B43A5758E548F000302000000220000000912000000091A000000010000000000000000000000000000000302000000220000000A120000000A1A0000000100000..........................................
3. What did you see instead (Required)
4. What is your TiFlash version? (Required)

V8.5.4

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the SQL reproduction on TiFlash V8.5.4 and inspect the failure reported at RaftCommands.cpp:506. Trace the integer decoding involving pk_type INT64 and schema_snapshot column mappings; done means the ALTER TABLE operation no longer produces the overflow error or repeated restarts.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.