debezium / debezium/dbz

Discrepancy between supplied configuration and runtime configuration [DBZ-8097]

Open
#997 0 comments 0 reactions 0 assignees View on GitHub
component/embedded-engine migrated-from-jira type/bug
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-8097](https://issues.redhat.com/browse/DBZ-8097)

In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
h1. Bug report

Debezium Server runtime configuration has discrepant key and value converters in

 
{code:java}
| key.converter.value.converter = io.apicurio.registry.utils.converter.AvroConverter
| key.converter = org.apache.kafka.connect.json.JsonConverter
| key.converter.key.converter = io.apicurio.registry.utils.converter.AvroConverter
| header.converter.key.converter = io.apicurio.registry.utils.converter.AvroConverter
| header.converter.value.converter = io.apicurio.registry.utils.converter.AvroConverter
| value.converter.value.converter = io.apicurio.registry.utils.converter.AvroConverter
| value.converter = org.apache.kafka.connect.json.JsonConverter
| value.converter.key.converter = io.apicurio.registry.utils.converter.AvroConverter{code}
 
h2. What Debezium connector do you use and what version?

Debezium -Server, -Pubsub 2.6.0.Final
h2. What is the connector configuration?

 
{code:java}
debezium.source.connector.class=io.debezium.connector.postgresql.PostgresConnector
debezium.source.snapshot.mode=never
debezium.source.publication.autocreate.mode=disabled
debezium.source.money.fraction.digits=8
debezium.source.offset.storage.file.filename=${OFFSET_FILE}
debezium.source.tombstones.on.delete=false
debezium.source.plugin.name=pgoutput
debezium.source.name=test
debezium.source.publication.name=${PUBLICATION_NAME}
debezium.source.slot.name=${SLOT_NAME}
#debezium.source.max.batch.size=2000
#debezium.source.max.queue.size=1000000
debezium.source.database.hostname=${DATABASE_HOSTNAME}
debezium.source.database.port=${DATABASE_PORT}
debezium.source.database.dbname=${DATABASE_DBNAME}
debezium.source.database.user=${DATABASE_USER}
debezium.source.database.password=${DATABASE_PASSWORD}
debezium.source.table.include.list=${TABLES}
debezium.source.topic.prefix=test
debezium.sink.type=pubsub
debezium.sink.pubsub.project.id=${PROJECT_ID}
debezium.sink.pubsub.wait.message.delivery.timeout.ms=90000
debezium.sink.pubsub.batch.element.count.threshold=100
debezium.sink.pubsub.ordering.enabled=true
quarkus.log.console.json=false
errors.max.retries=10
quarkus.http.port=8081
debezium.format.key=avro
debezium.format.value=avro
debezium.format.schema.name.adjustment.mode=avro
debezium.format.key.converter=io.apicurio.registry.utils.converter.AvroConverter
debezium.format.value.converter=io.apicurio.registry.utils.converter.AvroConverter
debezium.format.key.apicurio.registry.url=${SCHEMA_REGISTRY_URL}/apis/registry/v2
debezium.format.key.apicurio.registry.auto-register=true
debezium.format.key.apicurio.registry.find-latest=true
debezium.format.value.apicurio.registry.url=${SCHEMA_REGISTRY_URL}/apis/registry/v2
debezium.format.value.apicurio.registry.auto-register=true
debezium.format.value.apicurio.registry.find-latest=true{code}
 
h2. What is the captured database version and mode of deployment?

PostgreSQL 15.7-alpine 

Local Environment Docker compose 
h2. What behavior do you expect?

I would expect the key and value converters to be aligned with supplied configuration and not show any redundant configuration.
h2. What behavior do you see?

Discrepancies between application.properties and runtime configuration
h2. Do you see the same behaviour using the latest released Debezium version?

{color:#151515}Don't Do you have the connector logs, ideally from start till finish?{color}

(You might be asked later to provide [DEBUG/TRACE|https://debezium.io/documentation/reference/stable/operations/logging.html] level log)

h2. How to reproduce the issue using our [tutorial|https://github.com/debezium/debezium-examples/tree/main/tutorial] deployment?

h1. Feature request or enhancement

For feature requests or enhancements, provide this information, please:
h2. Which use case/requirement will be addressed by the proposed feature?

h2. Implementation ideas (optional)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the discrepancy with the supplied application.properties in the Debezium Server 2.6.0.Final local Docker Compose deployment, then compare the supplied and runtime converter settings. Confirm the expected result by ensuring the runtime key and value converters match the configuration without redundant entries.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker-compose, java, postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.