Discrepancy between supplied configuration and runtime configuration [DBZ-8097]
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
Migrated from [DBZ-8097](https://issues.redhat.com/browse/DBZ-8097)
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
h1. Bug report
Debezium Server runtime configuration has discrepant key and value converters in
{code:java}
| key.converter.value.converter = io.apicurio.registry.utils.converter.AvroConverter
| key.converter = org.apache.kafka.connect.json.JsonConverter
| key.converter.key.converter = io.apicurio.registry.utils.converter.AvroConverter
| header.converter.key.converter = io.apicurio.registry.utils.converter.AvroConverter
| header.converter.value.converter = io.apicurio.registry.utils.converter.AvroConverter
| value.converter.value.converter = io.apicurio.registry.utils.converter.AvroConverter
| value.converter = org.apache.kafka.connect.json.JsonConverter
| value.converter.key.converter = io.apicurio.registry.utils.converter.AvroConverter{code}
h2. What Debezium connector do you use and what version?
Debezium -Server, -Pubsub 2.6.0.Final
h2. What is the connector configuration?
{code:java}
debezium.source.connector.class=io.debezium.connector.postgresql.PostgresConnector
debezium.source.snapshot.mode=never
debezium.source.publication.autocreate.mode=disabled
debezium.source.money.fraction.digits=8
debezium.source.offset.storage.file.filename=${OFFSET_FILE}
debezium.source.tombstones.on.delete=false
debezium.source.plugin.name=pgoutput
debezium.source.name=test
debezium.source.publication.name=${PUBLICATION_NAME}
debezium.source.slot.name=${SLOT_NAME}
#debezium.source.max.batch.size=2000
#debezium.source.max.queue.size=1000000
debezium.source.database.hostname=${DATABASE_HOSTNAME}
debezium.source.database.port=${DATABASE_PORT}
debezium.source.database.dbname=${DATABASE_DBNAME}
debezium.source.database.user=${DATABASE_USER}
debezium.source.database.password=${DATABASE_PASSWORD}
debezium.source.table.include.list=${TABLES}
debezium.source.topic.prefix=test
debezium.sink.type=pubsub
debezium.sink.pubsub.project.id=${PROJECT_ID}
debezium.sink.pubsub.wait.message.delivery.timeout.ms=90000
debezium.sink.pubsub.batch.element.count.threshold=100
debezium.sink.pubsub.ordering.enabled=true
quarkus.log.console.json=false
errors.max.retries=10
quarkus.http.port=8081
debezium.format.key=avro
debezium.format.value=avro
debezium.format.schema.name.adjustment.mode=avro
debezium.format.key.converter=io.apicurio.registry.utils.converter.AvroConverter
debezium.format.value.converter=io.apicurio.registry.utils.converter.AvroConverter
debezium.format.key.apicurio.registry.url=${SCHEMA_REGISTRY_URL}/apis/registry/v2
debezium.format.key.apicurio.registry.auto-register=true
debezium.format.key.apicurio.registry.find-latest=true
debezium.format.value.apicurio.registry.url=${SCHEMA_REGISTRY_URL}/apis/registry/v2
debezium.format.value.apicurio.registry.auto-register=true
debezium.format.value.apicurio.registry.find-latest=true{code}
h2. What is the captured database version and mode of deployment?
PostgreSQL 15.7-alpine
Local Environment Docker compose
h2. What behavior do you expect?
I would expect the key and value converters to be aligned with supplied configuration and not show any redundant configuration.
h2. What behavior do you see?
Discrepancies between application.properties and runtime configuration
h2. Do you see the same behaviour using the latest released Debezium version?
{color:#151515}Don't Do you have the connector logs, ideally from start till finish?{color}
(You might be asked later to provide [DEBUG/TRACE|https://debezium.io/documentation/reference/stable/operations/logging.html] level log)
h2. How to reproduce the issue using our [tutorial|https://github.com/debezium/debezium-examples/tree/main/tutorial] deployment?
h1. Feature request or enhancement
For feature requests or enhancements, provide this information, please:
h2. Which use case/requirement will be addressed by the proposed feature?
h2. Implementation ideas (optional)
Contributor guide
Research direction
Start by reproducing the discrepancy with the supplied application.properties in the Debezium Server 2.6.0.Final local Docker Compose deployment, then compare the supplied and runtime converter settings. Confirm the expected result by ensuring the runtime key and value converters match the configuration without redundant entries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker-compose, java, postgresql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100