Oracle Connector: Missing CDC events
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
## Bug report
**What Debezium connector do you use and what version?**
`Debezium 3.4.0`
---
**What is the connector configuration?**
selected connector settings:
```
"connector.class": "io.debezium.connector.oracle.OracleConnector",
"key.converter": "io.confluent.connect.avro.AvroConverter",
"key.converter.schema.registry.url": "[removed]",
"value.converter": "io.confluent.connect.avro.AvroConverter",
"value.converter.schema.registry.url": "[removed]",
"schema.name.adjustment.mode": "avro",
"database.query.timeout.ms": 0,
"log.mining.strategy": "hybrid",
"log.mining.transaction.retention.ms": 50000000,
"log.mining.query.filter.mode": "in",
"log.mining.batch.size.max": 200000,
"log.mining.batch.size.min": 40000,
"log.mining.batch.size.default": 40000,
"internal.log.mining.log.query.max.retries": "10",
"internal.log.mining.sql.relaxed.quote.detection": "true",
"lob.enabled": "false",
"unavailable.value.placeholder": "previous_value_unchanged",
"time.precision.mode": "connect",
"tasks.max": "1",
"post.processors": "reselector",
"reselector.type": "io.debezium.processors.reselect.ReselectColumnsPostProcessor",
"reselector.reselect.columns.include.list": "[removed]",
"reselector.reselect.unavailable.values": "true",
"reselector.reselect.null.values": "true"
```
---
**What is the captured database version and mode of deployment?**
`Oracle 19c` on premise
---
**What behavior do you expect?**
All events are produced to the Kafka topic.
---
**What behavior do you see?**
Some events are missing in the Kafka topic. The missing events are really rare, depending on the table, one minssing in half a million events, or rarer.
I suspect that it could be related to the reselector, because only tables on the reselector include list are affected. We have VARCHAR2 columns with extended string size on our database, therefore a lot of tables / columns are on the reselector include list.
This is what one specific transaction looked like:
START
UPDATE: Old row was updated. This was captured by the connector and produced to the Kafka topic.
INTERNAL
INSERT: New row was inserted. Extended VARCHAR2 columns were inserted with LM_EMPTY_STRING. This was not produced to the Kafka topic.
UPDATE: New row was updated. Extended VARCHAR2 columns were updated to NULL. This was not produced to the Kafka topic.
COMMIT
---
**Do you see the same behaviour using the latest released Debezium version?**
Not applicable. We are not using the latest version.
---
**Do you have the connector logs, ideally from start till finish?**
I have access to the logs, but there is nothing suspicious in them, only commits. We don't have DEBUG/TRACE because of the amount it generates.
---
**How to reproduce the issue using our [tutorial](https://github.com/debezium/debezium-examples/tree/main/tutorial) deployment?**
I have not tried to reproduce it.
Contributor guide
Assessment
This issue has not been assessed yet.