Aiven-Open / Aiven-Open/opensearch-connector-for-apache-kafka

Topics regex failure

Aperta
#140 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Java
Stelle
95
Fork
61
Merge medio
11h 52m
PR unite (30g)
7

Descrizione

Using the topic regex feature crashes connector.

Details:
topic regex set to `.*`, topics is left empty, using the following regex router mapping topic to index
```
"transforms.topic-index-mapping.regex": ".*",
"transforms.topic-index-mapping.replacement": "metrics-iot-$0",
"transforms.topic-index-mapping.type": "org.apache.kafka.connect.transforms.RegexRouter"
```
yields the following stacktrace
```
Selected Task Stacktrace

org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:611)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:333)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:234)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:203)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:188)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to bulk processing after total of 1 attempt(s)
at io.aiven.kafka.connect.opensearch.RetryUtil.callWithRetry(RetryUtil.java:137)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.execute(BulkProcessor.java:367)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.call(BulkProcessor.java:356)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.call(BulkProcessor.java:337)
... 4 more
Caused by: org.apache.kafka.connect.errors.ConnectException: Bulk request failed: OpenSearchException[OpenSearch exception [type=mapper_parsing_exception, reason=object mapping for [properties.topics] tried to parse field [topics] as object, but found a concrete value]]
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.handleMalformedDoc(BulkProcessor.java:456)
at io.aiven.kafka.connect.opensearch.BulkProcessor$BulkTask.lambda$execute$0(BulkProcessor.java:380)
at io.aiven.kafka.connect.opensearch.RetryUtil.callWithRetry(RetryUtil.java:119)
... 7 more

```

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.