ERROR 9014 (HY000): TiFlash backfill index failed: Code: 0
Open
Nobody has claimed this yet.
contribution
type/bug
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
- minio server /tmp/tidb
- tiup playground
tiup playground nightly --mode=tidb-cse --tag cse-test
--db.binpath=${tidb-server-path}
--kv.binpath=${tikv-server-path}
--pd.binpath=${pd-server-path}
--tiflash.binpath=${tiflash-server-path} --tiflash.compute 1 --tikv.columnar - create vector table
CREATE DATABASE IF NOT EXISTS vector_db_test;
USE vector_db_test;
mysql> show create table vector_bench_test\G
*************************** 1. row ***************************
Table: vector_bench_test
Create Table: CREATE TABLE `vector_bench_test` (
`id` bigint(20) NOT NULL /*T![auto_rand] AUTO_RANDOM(5) */,
`embedding` vector(768) NOT NULL,
PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */,
VECTOR INDEX `idx_embedding`((VEC_COSINE_DISTANCE(`embedding`)))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin
1 row in set (0.03 sec)
- Vectorbench for inital dataset and query test.
2. What did you expect to see? (Required)
No error output, Vector Index can be used on the Vector Search.
3. What did you see instead (Required)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+
| TIDB_DATABASE | TIDB_TABLE | TIDB_PARTITION | TABLE_ID | BELONGING_TABLE_ID | COLUMN_NAME | INDEX_NAME | COLUMN_ID | INDEX_ID | INDEX_KIND | ROWS_STABLE_INDEXED | ROWS_STABLE_NOT_INDEXED | ROWS_DELTA_INDEXED | ROWS_DELTA_NOT_INDEXED | ERROR_MESSAGE | TIFLASH_INSTANCE |
+---------------+-------------------+----------------+----------+--------------------+-------------+--------------+-----------+----------+------------+---------------------+-------------------------+--------------------+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+
| vector_test | docs | NULL | 143 | -1 | emb | idx_emb | 3 | 1 | Vector | 0 | 0 | 0 | 0 | | store-c18476da |
| vector_test | vector_bench_test | NULL | 150 | -1 | embedding | vector_index | 2 | 1 | Vector | 0 | 1000000 | 0 | 0 | Code: 0, e.displayText() = DB::Exception: cannot load checksum framed data from tiflash/s138/data/ks_1_t_150/dmf_73/2.dat (errno = 115): (while reading from DTFile: s3://s138/data/ks_1_t_150/dmf_73, column: 2:): while building stable index for index_ids=[1], e.what() = DB::Exception | store-c18476da |
+---------------+-------------------+----------------+----------+--------------------+-------------+--------------+-----------+----------+------------+---------------------+-------------------------+--------------------+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+
2 rows in set (0.03 sec)
mysql> alter table vector_bench_test add VECTOR INDEX vector_index((VEC_COSINE_DISTANCE(embedding)));
ERROR 9014 (HY000): TiFlash backfill index failed: Code: 0, e.displayText() = DB::Exception: cannot load checksum framed data from tiflash/s138/data/ks_1_t_154/dmf_99/2.dat (errno = 115): (while reading from DTFile: s3://s138/data/ks_1_t_154/dmf_99, column: 2:): while building stable index for index_ids=[3], e.what() = DB::Exception
mysql> alter table vector_bench_test add VECTOR INDEX vector_index((VEC_COSINE_DISTANCE(embedding)));
ERROR 9014 (HY000): TiFlash backfill index failed: Code: 0, e.displayText() = DB::Exception: cannot load checksum framed data from tiflash/s138/data/ks_1_t_154/dmf_100/2.dat (errno = 115): (while reading from DTFile: s3://s138/data/ks_1_t_154/dmf_100, column: 2:): while building stable index for index_ids=[4], e.what() = DB::Exception
4. What is your TiFlash version? (Required)
mysql> select * from cluster_info
-> ;
+-----------------+------------------+------------------+-------------------------+------------------------------------------+----------------------+---------------------+-----------+
| TYPE | INSTANCE | STATUS_ADDRESS | VERSION | GIT_HASH | START_TIME | UPTIME | SERVER_ID |
+-----------------+------------------+------------------+-------------------------+------------------------------------------+----------------------+---------------------+-----------+
| tidb | 172.23.0.2:4000 | 172.23.0.2:10080 | 7.5.6-serverless | cd5a2c789dcafb31be003e7271da246508b5f2f7 | 2025-12-16T15:46:32Z | 13h58m46.123246514s | 1282 |
| pd | 172.23.0.2:2379 | 172.23.0.2:2379 | 8.5.0-serverless | 6e2a4d2024251e23168d1ce356eea19c05e10089 | 2025-12-16T15:46:12Z | 13h59m6.123256162s | 0 |
| tikv | 172.23.0.2:20160 | 172.23.0.2:20180 | 7.5.0 | c85a029d3ae08685d89509e99204c7a6bb268b59 | 2025-12-16T15:46:14Z | 13h59m4.123256988s | 0 |
| tiflash_write | 172.23.0.2:3930 | 172.23.0.2:20292 | 8.5.0-cse-96-g9568c16e5 | 9568c16e597211acd9809d6c4429b09b1a0d6ffc | 2025-12-16T15:46:50Z | 13h58m28.12325777s | 0 |
| tiflash_compute | 172.23.0.2:3931 | 172.23.0.2:20293 | 8.5.0-cse-96-g9568c16e5 | 9568c16e597211acd9809d6c4429b09b1a0d6ffc | 2025-12-16T15:46:50Z | 13h58m28.123258694s | 0 |
+-----------------+------------------+------------------+-------------------------+------------------------------------------+----------------------+---------------------+-----------+
5 rows in set (0.06 sec)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided minio server and tiup playground setup, create the vector table, load the initial dataset, and run the ALTER TABLE command to reproduce the TiFlash backfill failure. Inspect the TiFlash error output around the checksum-framed DTFile read; done means vector index backfill completes without ERROR 9014 and the index can be used for vector search.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100