Aiven-Open / Aiven-Open/http-connector-for-apache-kafka
Flatten transform causes this Connector to fail
- Langage dominant
- Java
- Étoiles
- 119
- Forks
- 63
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
The HTTP connector I had configured for my Aiven Kafka service started failing after I configured a 'Flatten' transform (`org.apache.kafka.connect.transforms.Flatten$Value`) on the connector.
I kept getting the following exception:
```
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:611)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:333)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:234)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:203)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:188)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.kafka.connect.errors.DataException: Record value must be String, Schema Struct or HashMap, but class java.util.LinkedHashMap is given
at io.aiven.kafka.connect.http.converter.RecordValueConverter.convert(RecordValueConverter.java:44)
at io.aiven.kafka.connect.http.recordsender.BatchRecordSender.createRequestBody(BatchRecordSender.java:77)
at io.aiven.kafka.connect.http.recordsender.BatchRecordSender.send(BatchRecordSender.java:60)
at io.aiven.kafka.connect.http.HttpSinkTask.put(HttpSinkTask.java:97)
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:581)
... 10 more
```
I emphasize that this only occurred after applying the "Flatten" transform on the value.
Here is the configuration I was using:
```
{
"http.url": "PLACEHOLDER_URL",
"batching.enabled": "true",
"name": "clickhouse_player_view_events",
"http.authorization.type": "static",
"batch.max.size": "2500",
"transforms": "transform-1",
"connector.class": "io.aiven.kafka.connect.http.HttpSinkConnector",
"http.headers.authorization": "PLACEHOLDER_AUTHORIZATION_HEADER",
"tasks.max": "2",
"http.headers.content.type": "application/json",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"errors.log.enable": "true",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"errors.log.include.messages": "true",
"topics": "player_analytics",
"request.method": "POST",
"retry.on.status.codes": "400-500",
"ssl.enabled": "true",
"value.converter.schemas.enable": "false",
"transforms.transform-1.delimiter": ".",
"transforms.transform-1.type": "org.apache.kafka.connect.transforms.Flatten$Value"
}
```
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.