Aiven-Open / Aiven-Open/opensearch-connector-for-apache-kafka
Topics regex failure
- Vorherrschende Sprache
- Java
- Sterne
- 95
- Forks
- 61
- Ø Merge
- 11 Std. 52 Min.
- Gemergte PRs (30 T.)
- 7
Beschreibung
Using the topic regex feature crashes connector.
Details:
topic regex set to `.*`, topics is left empty, using the following regex router mapping topic to index
```
"transforms.topic-index-mapping.regex": ".*",
"transforms.topic-index-mapping.replacement": "metrics-iot-$0",
"transforms.topic-index-mapping.type": "org.apache.kafka.connect.transforms.RegexRouter"
```
yields the following stacktrace
```
Selected Task Stacktrace
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:611)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:333)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:234)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:203)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:188)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to bulk processing after total of 1 attempt(s)
at io.aiven.kafka.connect.opensearch.RetryUtil.callWithRetry(RetryUtil.java:137)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.execute(BulkProcessor.java:367)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.call(BulkProcessor.java:356)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.call(BulkProcessor.java:337)
... 4 more
Caused by: org.apache.kafka.connect.errors.ConnectException: Bulk request failed: OpenSearchException[OpenSearch exception [type=mapper_parsing_exception, reason=object mapping for [properties.topics] tried to parse field [topics] as object, but found a concrete value]]
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.handleMalformedDoc(BulkProcessor.java:456)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.lambda$execute$0(BulkProcessor.java:380)
at io.aiven.kafka.connect.opensearch.RetryUtil.callWithRetry(RetryUtil.java:119)
... 7 more
```
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.