Panic found event_broker `get table info from schemaStore failed`
Open
Nobody has claimed this yet.
type/bug
- Dominant language
- Go
- Stars
- 56
- Forks
- 63
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 34
Description
What did you do?
run the ci
What did you expect to see?
pass
What did you see instead?
failed the test, and found panic
[2025/03/27 11:08:11.200 +08:00] [PANIC] [event_broker.go:904] ["get table info from schemaStore failed"] [error="no version found"] [tableID=408] [startTs=456928749695008795] [stack="github.com/pingcap/ticdc/pkg/eventservice.(*eventBroker).addDispatcher\n\t/home/jenkins/agent/workspace/pingcap/ticdc/pull_cdc_mysql_integration_heavy/ticdc/pkg/eventservice/event_broker.go:904\ngithub.com/pingcap/ticdc/pkg/eventservice.(*eventService).registerDispatcher\n\t/home/jenkins/agent/workspace/pingcap/ticdc/pull_cdc_mysql_integration_heavy/ticdc/pkg/eventservice/event_service.go:145\ngithub.com/pingcap/ticdc/pkg/eventservice.(*eventService).Run\n\t/home/jenkins/agent/workspace/pingcap/ticdc/pull_cdc_mysql_integration_heavy/ticdc/pkg/eventservice/event_service.go:97\ngithub.com/pingcap/ticdc/server.(*server).Run.(*server).Run.func2.func3\n\t/home/jenkins/agent/workspace/pingcap/ticdc/pull_cdc_mysql_integration_heavy/ticdc/server/server.go:241\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/go/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78"]
goroutine 1525 [running]:
go.uber.org/zap/zapcore.CheckWriteAction.OnWrite(0x2, 0xc0071cfa00, {0x46?, 0xc009572000?, 0x35a?})
/go/pkg/mod/go.uber.org/zap@v1.27.0/zapcore/entry.go:196 +0x9f
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0071cfa00, {0xc000311380, 0x3, 0x3})
/go/pkg/mod/go.uber.org/zap@v1.27.0/zapcore/entry.go:262 +0x3ad
go.uber.org/zap.(*Logger).Panic(0xc0032cbe00, {0xbf88276, 0x26}, {0xc000311380, 0x3, 0x3})
/go/pkg/mod/go.uber.org/zap@v1.27.0/logger.go:285 +0x68
github.com/pingcap/log.Panic({0xbf88276, 0x26}, {0xc000311380, 0x3, 0x3})
/go/pkg/mod/github.com/pingcap/log@v1.1.1-0.20241212030209-7e3ff8601a2a/global.go:54 +0x72
github.com/pingcap/ticdc/pkg/eventservice.(*eventBroker).addDispatcher(0xc0013b6780, {0xd068700, 0xc000fc4930})
/home/jenkins/agent/workspace/pingcap/ticdc/pull_cdc_mysql_integration_heavy/ticdc/pkg/eventservice/event_broker.go:904 +0x1a49
github.com/pingcap/ticdc/pkg/eventservice.(*eventService).registerDispatcher(0xc001378140, {0xd02a890, 0xc001378460}, {0xd068700, 0xc000fc4930})
/home/jenkins/agent/workspace/pingcap/ticdc/pull_cdc_mysql_integration_heavy/ticdc/pkg/eventservice/event_service.go:145 +0x1ab
github.com/pingcap/ticdc/pkg/eventservice.(*eventService).Run(0xc001378140, {0xd02a890, 0xc001378460})
/home/jenkins/agent/workspace/pingcap/ticdc/pull_cdc_mysql_integration_heavy/ticdc/pkg/eventservice/event_service.go:97 +0x309
github.com/pingcap/ticdc/server.(*server).Run.(*server).Run.func2.func3()
/home/jenkins/agent/workspace/pingcap/ticdc/pull_cdc_mysql_integration_heavy/ticdc/server/server.go:241 +0x36f
golang.org/x/sync/errgroup.(*Group).Go.func1()
/go/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78 +0x92
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 587
/go/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:75 +0x125
Versions of the cluster
Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):
(paste TiDB cluster version here)
Upstream TiKV version (execute tikv-server --version):
(paste TiKV version here)
TiCDC version (execute cdc version):
(paste TiCDC version here)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at pkg/eventservice/event_broker.go:904 in addDispatcher, then trace registration through event_service.go:145 and event_service.go:97. Reproduce the mysql integration CI failure from the linked Jenkins run and investigate the schemaStore lookup for tableID 408; done means the CI test passes without the reported panic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100