TimeOutException: Topic not present in metadata after 10000ms [DBZ-8578]
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
Migrated from [DBZ-8578](https://issues.redhat.com/browse/DBZ-8578)
h1. Bug report
For bug reports, provide this information, please:
h2. What Debezium connector do you use and what version?
debezium server (3.0.2-final) with MySqlConnector
h2. What is the connector configuration?
```
# sink config
debezium.sink.type=kafka
debezium.sink.kafka.producer.bootstrap.servers=XXX
debezium.sink.kafka.producer.key.serializer=org.apache.kafka.common.serialization.StringSerializer
debezium.sink.kafka.producer.value.serializer=org.apache.kafka.common.serialization.StringSerializer
debezium.sink.kafka.producer.security.protocol=SASL_SSL
debezium.sink.kafka.producer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='XXX' password='XXX';
debezium.sink.kafka.producer.sasl.mechanism=PLAIN
debezium.sink.kafka.producer.client.dns.lookup=use_all_dns_ips
debezium.sink.kafka.producer.client=PLAIN
debezium.sink.kafka.producer.client.id=vmCDC-SX
debezium.sink.kafka.producer.compression.type=lz4
debezium.sink.kafka.producer.creation.enable=false
# source config
debezium.source.connector.class=io.debezium.connector.mysql.MySqlConnector
debezium.source.database.hostname=vmCDC.local
debezium.source.database.port=3306
debezium.source.database.user=XXX
debezium.source.database.password=XXX
debezium.source.database.server.id=1
debezium.source.topic.prefix=XXX.test.events
debezium.source.database.include.list=db
debezium.source.column.exclude.list=db.tpltsk_issuestate.IconData
debezium.source.table.include.list=db.rel_country,db.rel_municipality,db.rel_company,db.rel_address,db.rel_relation,db.rel_supplieractivity,db.rel_communicationchannel,db.fin_bankaccount,db.rel_supplieractivitytype,db.sec_user,db.rel_communication,db.rel_financial,db.tpltsk_issuestate,db.tpltsk_issue
debezium.source.offset.storage.file.filename=data/offsets.dat
debezium.source.schema.history.internal.kafka.bootstrap.servers=XXX
debezium.source.schema.history.internal.kafka.sasl.mechanism=PLAIN
debezium.source.schema.history.internal.kafka.topic=XXX.test.events-internal
debezium.source.schema.history.internal.kafka.security.protocol=SASL_SSL
debezium.source.schema.history.internal.kafka.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='XXX' password='XXX';
debezium.source.schema.history.internal.consumer.security.protocol=SASL_SSL
debezium.source.schema.history.internal.consumer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='XXX' password='XXX';
debezium.source.schema.history.internal.consumer.sasl.mechanism=PLAIN
debezium.source.schema.history.internal.producer.security.protocol=SASL_SSL
debezium.source.schema.history.internal.producer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='XXX' password='XXX';
debezium.source.schema.history.internal.producer.sasl.mechanism=PLAIN
debezium.source.include.schema.changes=false
debezium.source.topic.creation.default.compression.type=lz4
debezium.source.topic.creation.enable=false
debezium.source.provide.transaction.metadata=true
debezium.source.tombstones.on.delete=false
debezium.source.snapshot.mode=no_data
# signaling => listens to kafka topic test-signal and executes the commands
debezium.source.signal.enabled.channels=kafka
debezium.source.signal.kafka.topic=XXX-signal
debezium.source.signal.kafka.bootstrap.servers=XXX
debezium.source.signal.kafka.sasl.mechanism=PLAIN
debezium.source.signal.kafka.security.protocol=SASL_SSL
debezium.source.signal.kafka.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='XXX' password='XXX';
debezium.source.signal.consumer.security.protocol=SASL_SSL
debezium.source.signal.consumer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='XXX' password='XXX';
debezium.source.signal.consumer.sasl.mechanism=PLAIN
debezium.source.signal.consumer.client.id=signal-listener-for-kafka-agent
debezium.source.signal.producer.security.protocol=SASL_SSL
debezium.source.signal.producer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='XXX' password='XXX';
debezium.source.signal.producer.sasl.mechanism=PLAIN
# format
debezium.format.key=json
debezium.format.key.schemas.enable=false
debezium.format.value=json
debezium.format.value.schemas.enable=false
debezium.format.header=json
# transformations
debezium.transforms=beginFilter,endFilter,reroute,tenantInsert
debezium.transforms.reroute.type=org.apache.kafka.connect.transforms.RegexRouter
debezium.transforms.reroute.regex=XXX.test.events.([^.]*).(.*)
debezium.transforms.reroute.replacement=XXX.test.events
debezium.transforms.beginFilter.topic.regex=XXX.test.events.transaction
debezium.transforms.beginFilter.type=io.debezium.transforms.Filter
debezium.transforms.beginFilter.language=jsr223.groovy
debezium.transforms.beginFilter.condition=value.status != 'BEGIN'
debezium.transforms.endFilter.topic.regex=XXX.test.events.transaction
debezium.transforms.endFilter.type=io.debezium.transforms.Filter
debezium.transforms.endFilter.language=jsr223.groovy
debezium.transforms.endFilter.condition=!(value.status == 'END' && value.event_count == 0)
debezium.transforms.tenantInsert.type=org.apache.kafka.connect.transforms.InsertField$Value
debezium.transforms.tenantInsert.static.field=tenantId
debezium.transforms.tenantInsert.static.value=XXX
# logging
quarkus.log.console.json=false
quarkus.log.level=INFO
quarkus.http.port=8080
```
h2. What is the captured database version and mode of deployment?
(E.g. on-premises, with a specific cloud provider, etc.)
on premise database : mysql => Version 5.6
h2. What behavior do you expect?
This not to happen
h2. What behavior do you see?
This works fine for 99% but sometimes I get this exception:
!image-2025-01-15-19-57-07-479.png!
h2. Do you see the same behaviour using the latest released Debezium version?
(Ideally, also verify with latest Alpha/Beta/CR version)
i'm using version 3.0.2 of debezium server and version 3 is latest release
Contributor guide
Research direction
Start by reproducing the intermittent exception with Debezium Server 3.0.2, MySQL 5.6, and the provided Kafka sink and source configuration. Trace the path associated with “Topic not present in metadata” and verify that the configured topics become available reliably without the timeout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka, mysql
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100