Aiven-Open / Aiven-Open/opensearch-connector-for-apache-kafka

behavior.on.version.conflict=warn still throws errors

Đang mở
#428 3 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Java
Star
95
Fork
61
Merge trung bình
11 giờ 52 phút
Pull request đã merge (30 ngày)
7

Mô tả

Hello 👋, I recently updated our connector to 4.1.0 and started getting these 👇 errors when I run a backfill. The version conflicts should be ignored and just log so the config is set to `"behavior.on.version.conflict": "warn",` but the connector still gets exception instead of warn log. Also one of the JMX metric we use to indicate if the connector is in fact running is `jmx.kafka.connect.total_record_failures` which also shows `34.8k`.
Am I missing some something or is this a bug? I saw https://github.com/Aiven-Open/opensearch-connector-for-apache-kafka/pull/413 and https://github.com/Aiven-Open/opensearch-connector-for-apache-kafka/pull/419 were fixed in V4.1.0 so this should have worked.

ERROR:
```
Jun 26 13:23:40.360 prod-kc-os-sink-track info java.lang.Exception: Encountered a version conflict when executing batch 601 of 49 records. Reporting this error to the errant record reporter and will keep an existing record. Rest status: 409, Action id: 48, Error message: [9103882]: version conflict, current version [2660808] is higher or equal to the one provided [2660808]
Jun 26 13:23:40.360 prod-kc-os-sink-track info [2026-06-26 07:53:40,353] ERROR Error encountered in task prod-kc-os-sink-track-0. Executing stage 'TASK_PUT' with class 'org.apache.kafka.connect.sink.SinkTask', where consumed record is {topic='event.backup.prod.etl.tracks', partition=0, offset=2660808, timestamp=1782459555748, timestampType=CreateTime}. (org.apache.kafka.connect.runtime.errors.LogReporter)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at java.base/java.lang.Thread.run(Unknown Source)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.call(BulkProcessor.java:342)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.call(BulkProcessor.java:361)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.execute(BulkProcessor.java:389)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.RetryUtil.callWithRetry(RetryUtil.java:97)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.lambda$execute$1(BulkProcessor.java:402)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.handleVersionConflict(BulkProcessor.java:439)
Jun 26 13:23:40.360 prod-kc-os-sink-track info at io.aiven.kafka.connect.opensearch.bulk.BulkProcessor$BulkTask.sendToErrantRecordReporter(BulkProcessor.java:373)
```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start by tracing the version-conflict path in BulkProcessor.java, especially BulkTask.call, execute, handleVersionConflict, and sendToErrantRecordReporter at the lines shown in the stack trace. Check how behavior.on.version.conflict=warn is read and applied, then reproduce the backfill case and verify that the conflict is logged without throwing an exception or producing the reported task failure.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
java
Lĩnh vực
backend, distributed-systems
Loại issue
Lỗi
Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
57/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.