Aiven-Open / Aiven-Open/opensearch-connector-for-apache-kafka
behavior.on.version.conflict=warn still throws errors
- Dominant language
- Java
- Stars
- 95
- Forks
- 61
- Avg merge
- 11h 52m
- Merged PRs (30d)
- 7
Description
Hello 👋, I recently updated our connector to 4.1.0 and started getting these 👇 errors when I run a backfill. The version conflicts should be ignored and just log so the config is set to `"behavior.on.version.conflict": "warn",` but the connector still gets exception instead of warn log. Also one of the JMX metric we use to indicate if the connector is in fact running is `jmx.kafka.connect.total_record_failures` which also shows `34.8k`.
Am I missing some something or is this a bug? I saw https://github.com/Aiven-Open/opensearch-connector-for-apache-kafka/pull/413 and https://github.com/Aiven-Open/opensearch-connector-for-apache-kafka/pull/419 were fixed in V4.1.0 so this should have worked.
ERROR:
```
Jun 26 13:23:40.360 prod-kc-os-sink-track info java.lang.Exception: Encountered a version conflict when executing batch 601 of 49 records. Reporting this error to the errant record reporter and will keep an existing record. Rest status: 409, Action id: 48, Error message: [9103882]: version conflict, current version [2660808] is higher or equal to the one provided [2660808]
Jun 26 13:23:40.360 prod-kc-os-sink-track info [2026-06-26 07:53:40,353] ERROR Error encountered in task prod-kc-os-sink-track-0. Executing stage 'TASK_PUT' with class 'org.apache.kafka.connect.sink.SinkTask', where consumed record is {topic='event.backup.prod.etl.tracks', partition=0, offset=2660808, timestamp=1782459555748, timestampType=CreateTime}. (org.apache.kafka.connect.runtime.errors.LogReporter)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at java.base/java.lang.Thread.run(Unknown Source)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.call(BulkProcessor.java:342)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.call(BulkProcessor.java:361)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.execute(BulkProcessor.java:389)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.RetryUtil.callWithRetry(RetryUtil.java:97)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.lambda$execute$1(BulkProcessor.java:402)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.handleVersionConflict(BulkProcessor.java:439)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.sendToErrantRecordReporter(BulkProcessor.java:373)
```
Contributor guide
Assessment
This issue has not been assessed yet.