pingcap / pingcap/tiflash

flaky test: StorageDeltaMergeTest.RestoreAfterClearData

Open
#11,079 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component/storage type/bug
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

https://do.pingcap.net/jenkins/job/pingcap/job/tiflash/job/pull_unit_test/1/stages/log?nodeId=115

2. What did you expect to see? (Required)

All ut passed

3. What did you see instead (Required)
[2026/09/04 15:25:42.340 +08:00] [DEBUG] [DeltaMergeStore.cpp:180] ["Segment task pop from background task pool, segment=<segment_id=38 epoch=0 range=[31,9223372036854775807)> task=Split"] [source="keyspace=4294967295 table_id=1233"] [thread_id=6]
[2026/09/04 15:25:42.340 +08:00] [DEBUG] [Segment.cpp:2060] ["Split - SplitPhysical - Begin prepare, opt_split_point=(null)"] [source="keyspace=4294967295 table_id=1233 segment_id=1 epoch=6"] [thread_id=7]
[2026/09/04 15:25:42.340 +08:00] [INFO] [DeltaMergeStore_InternalSegment.cpp:138] ["Split - Begin, mode=Auto reason=Background safe_point=0 segment=<segment_id=38 epoch=0 range=[31,9223372036854775807) next_segment_id=0 delta_rows=0 delta_bytes=0 delta_deletes=0 stable_file=[dmf_28(v=0)] stable_rows=31 stable_bytes=1116 stable_cols=5 dmf_rows=31 dmf_bytes=1116 dmf_disk_bytes=612 dmf_packs=1>"] [source="keyspace=4294967295 table_id=1233"] [thread_id=6]
[2026/09/04 15:25:42.340 +08:00] [DEBUG] [Segment.cpp:2637] ["Begin segment getReadInfo <segment_id=1 epoch=6 range=[-9223372036854775808,31)>"] [source=" seg_id=1"] [thread_id=7]
[2026/09/04 15:25:42.340 +08:00] [DEBUG] [Segment.cpp:2060] ["Split - SplitPhysical - Begin prepare, opt_split_point=(null)"] [source="keyspace=4294967295 table_id=1233 segment_id=38 epoch=0"] [thread_id=6]
[2026/09/04 15:25:42.340 +08:00] [DEBUG] [Segment.cpp:2637] ["Begin segment getReadInfo <segment_id=38 epoch=0 range=[31,9223372036854775807)>"] [source=" seg_id=38"] [thread_id=6]
[2026/09/04 15:25:42.340 +08:00] [DEBUG] [Segment.cpp:2664] ["Finish segment getReadInfo, my_delta_index=<placed_rows=0 placed_deletes=0 tree_entries=0 tree_inserts=0 tree_deletes=0> fully_indexed=false read_ranges=[[-9223372036854775808,9223372036854775807)] snap={stable_rows=31 persisted_rows=0 persisted_dels=0 persisted_cfs=0 mem_rows=0 mem_dels=0 mem_cfs=0} <segment_id=38 epoch=0 range=[31,9223372036854775807)>"] [source=" seg_id=38"] [thread_id=6]
/home/jenkins/agent/workspace/pingcap/tiflash/pull_unit_test/tiflash/dbms/src/Storages/DeltaMerge/tests/gtest_dm_storage_delta_merge.cpp:885: Failure
Expected equality of these values:
  storage->getStore()->getSegmentsStats().size()
    Which is: 2
  1
[2026/09/04 15:25:42.340 +08:00] [DEBUG] [StableValueSpace.cpp:474] ["StableVS getInputStream start_ts=18446744073709551615 enable_handle_clean_read=false is_fast_mode=false enable_del_clean_read=false"] [source="keyspace=4294967295 table_id=1233 segment_id=38 epoch=0"] [thread_id=6]
[2026/09/04 15:25:42.340 +08:00] [ERROR] [Exception.cpp:93] ["std::exception. Code: 1001, type: testing::AssertionException, e.what() = /home/jenkins/agent/workspace/pingcap/tiflash/pull_unit_test/tiflash/dbms/src/Storages/DeltaMerge/tests/gtest_dm_storage_delta_merge.cpp:885: Failure\nExpected equality of these values:\n  storage->getStore()->getSegmentsStats().size()\n    Which is: 2\n  1"] [source="virtual void DB::DM::tests::StorageDeltaMergeTest_RestoreAfterClearData_Test::TestBody()"] [thread_id=1]
/home/jenkins/agent/workspace/pingcap/tiflash/pull_unit_test/tiflash/dbms/src/Storages/DeltaMerge/tests/gtest_dm_storage_delta_merge.cpp:892: Failure
Failed
4. What is your TiFlash version? (Required)

https://github.com/pingcap/tiflash/pull/11061/changes/6a2ba91e09f7cf20f5958275a30488c46707d7bb from https://github.com/pingcap/tiflash/pull/11061

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tiflash/dbms/src/Storages/DeltaMerge/tests/gtest_dm_storage_delta_merge.cpp at lines 885 and 892, focusing on StorageDeltaMergeTest.RestoreAfterClearData. Run this unit test and inspect the logged segment split and restore behavior around the failing segment-count assertions. Done means the test no longer flakes and its expected segment count passes consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.