ticdc panic at start
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 56
- Forks
- 63
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 34
Description
What did you do?
[2026/01/02 04:29:01.902 +00:00] [FATAL] [persist_storage.go:258] ["fail to initialize from kv snapshot"] [error="[tikv:9006]GC life time is shorter than transaction duration, transaction starts at 2025-12-31 04:26:55.556 +0000 UTC, GC safe point is 2025-12-31 04:26:55.556 +0000 UTC"] [errorVerbose="[tikv:9006]GC life time is shorter than transaction duration, transaction starts at 2025-12-31 04:26:55.556 +0000 UTC, GC safe point is 2025-12-31 04:26:55.556 +0000 UTC\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\tgithub.com/pingcap/errors@v0.11.5-0.20250523034308-74f78ae071ee/normalize.go:177\ngithub.com/pingcap/tidb/pkg/store/driver/error.ToTiDBErr\n\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20250415080739-a02630cc24cf/pkg/store/driver/error/error.go:155\ngithub.com/pingcap/tidb/pkg/store/driver/txn.extractKeyErr\n\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20250415080739-a02630cc24cf/pkg/store/driver/txn/error.go:166\ngithub.com/pingcap/tidb/pkg/store/driver/txn.(*tikvSnapshot).Get\n\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20250415080739-a02630cc24cf/pkg/store/driver/txn/snapshot.go:61\ngithub.com/pingcap/tidb/pkg/structure.(*TxStructure).HGet\n\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20250415080739-a02630cc24cf/pkg/structure/hash.go:45\ngithub.com/pingcap/tidb/pkg/meta.(*Mutator).checkDBExists\n\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20250415080739-a02630cc24cf/pkg/meta/meta.go:548\ngithub.com/pingcap/tidb/pkg/meta.(*Mutator).GetMetasByDBID\n\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20250415080739-a02630cc24cf/pkg/meta/meta.go:1088\ngithub.com/pingcap/ticdc/logservice/schemastore.persistSchemaSnapshot\n\tgithub.com/pingcap/ticdc/logservice/schemastore/disk_format.go:618\ngithub.com/pingcap/ticdc/logservice/schemastore.(*persistentStorage).initializeFromKVStorage\n\tgithub.com/pingcap/ticdc/logservice/schemastore/persist_storage.go:256\ngithub.com/pingcap/ticdc/logservice/schemastore.(*persistentStorage).initialize\n\tgithub.com/pingcap/ticdc/logservice/schemastore/persist_storage.go:240\ngithub.com/pingcap/ticdc/logservice/schemastore.newPersistentStorage\n\tgithub.com/pingcap/ticdc/logservice/schemastore/persist_storage.go:160\ngithub.com/pingcap/ticdc/logservice/schemastore.(*schemaStore).RegisterKeyspace\n\tgithub.com/pingcap/ticdc/logservice/schemastore/schema_store.go:466\ngithub.com/pingcap/ticdc/logservice/schemastore.(*schemaStore).Run\n\tgithub.com/pingcap/ticdc/logservice/schemastore/schema_store.go:284\ngithub.com/pingcap/ticdc/server.(*server).Run.(*server).Run.func3.func8\n\tgithub.com/pingcap/ticdc/server/server.go:334\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/sync@v0.17.0/errgroup/errgroup.go:93\nruntime.goexit\n\truntime/asm_amd64.s:1700"] [stack="github.com/pingcap/ticdc/logservice/schemastore.(*persistentStorage).initializeFromKVStorage\n\tgithub.com/pingcap/ticdc/logservice/schemastore/persist_storage.go:258\ngithub.com/pingcap/ticdc/logservice/schemastore.(*persistentStorage).initialize\n\tgithub.com/pingcap/ticdc/logservice/schemastore/persist_storage.go:240\ngithub.com/pingcap/ticdc/logservice/schemastore.newPersistentStorage\n\tgithub.com/pingcap/ticdc/logservice/schemastore/persist_storage.go:160\ngithub.com/pingcap/ticdc/logservice/schemastore.(*schemaStore).RegisterKeyspace\n\tgithub.com/pingcap/ticdc/logservice/schemastore/schema_store.go:466\ngithub.com/pingcap/ticdc/logservice/schemastore.(*schemaStore).Run\n\tgithub.com/pingcap/ticdc/logservice/schemastore/schema_store.go:284\ngithub.com/pingcap/ticdc/server.(*server).Run.(*server).Run.func3.func8\n\tgithub.com/pingcap/ticdc/server/server.go:334\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/sync@v0.17.0/errgroup/errgroup.go:93"]
What did you expect to see?
No response
What did you see instead?
Nan
Versions of the cluster
Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):
(paste TiDB cluster version here)
Upstream TiKV version (execute tikv-server --version):
(paste TiKV version here)
TiCDC version (execute cdc version):
(paste TiCDC version here)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in logservice/schemastore/persist_storage.go at initializeFromKVStorage and follow the stack into disk_format.go's persistSchemaSnapshot, then inspect schema_store.go's RegisterKeyspace entry point. Reproduce the startup failure with the reported GC lifetime condition if possible; done means TiCDC no longer panics during initialization and the failure has appropriate coverage or handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100