Aiven-Open / Aiven-Open/opensearch-connector-for-apache-kafka

Topics regex failure

Offen
#140 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Java
Sterne
95
Forks
61
Ø Merge
11 Std. 52 Min.
Gemergte PRs (30 T.)
7

Beschreibung

Using the topic regex feature crashes connector.

Details:
topic regex set to `.*`, topics is left empty, using the following regex router mapping topic to index
```
"transforms.topic-index-mapping.regex": ".*",
"transforms.topic-index-mapping.replacement": "metrics-iot-$0",
"transforms.topic-index-mapping.type": "org.apache.kafka.connect.transforms.RegexRouter"
```
yields the following stacktrace
```
Selected Task Stacktrace

org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:611)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:333)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:234)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:203)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:188)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to bulk processing after total of 1 attempt(s)
at io.aiven.kafka.connect.opensearch.RetryUtil.callWithRetry(RetryUtil.java:137)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.execute(BulkProcessor.java:367)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.call(BulkProcessor.java:356)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.call(BulkProcessor.java:337)
... 4 more
Caused by: org.apache.kafka.connect.errors.ConnectException: Bulk request failed: OpenSearchException[OpenSearch exception [type=mapper_parsing_exception, reason=object mapping for [properties.topics] tried to parse field [topics] as object, but found a concrete value]]
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.handleMalformedDoc(BulkProcessor.java:456)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.lambda$execute$0(BulkProcessor.java:380)
at io.aiven.kafka.connect.opensearch.RetryUtil.callWithRetry(RetryUtil.java:119)
... 7 more

```

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.