Aiven-Open / Aiven-Open/http-connector-for-apache-kafka
How can I get the connector to submit the follow curl?
- Dominant language
- Java
- Stars
- 119
- Forks
- 63
- PR merge metrics
- No merged PRs in 30d
Description
There are not enough examples of how to get the http connector to work. Best example I could find is https://redpanda.com/blog/unify-log-data-parseable-redpanda
Here's what I need to submit
```
curl -v --location-trusted -u : -H "strict_mode: true" \
-H "format: json" -H "jsonpaths: [\"$.name\", \"$.code\"]" \
-H "columns: city,tmp_id, id = tmp_id * 100" \
-T example2.json -XPUT \
http://someserver:<443>/api/test_db/table2/_stream_load
```
I need multiple headers to be submitted but the following won't work since http.headers.additional is a String.
```
{
"name": "http-sink-test",
"config": {
"connector.class": "io.aiven.kafka.connect.http.HttpSinkConnector",
"topics.regex": "m1",
"http.authorization.type": "oauth2",
"oauth2.client.id": "username",
"oauth2.client.secret": "password",
"http.headers.additional": [
{
"format": "json"
},
{
"jsonpaths": "[\"$.userid\",\"$.time\",\"$._time\",\"$.remote_user\",\"$.ip\",\"$.request\",\"$.status\",\"$.bytes\",\"$.referrer\",\"$.agent\"]"
}
],
"http.url": "http://https://6dihxzedg.cloud-app.celerdata.com/api/demo/visits/_stream_load",
"batching.enabled": true,
"batch.max.size": 2,
"tasks.max": "1",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter": "org.apache.kafka.connect.json.JsonConverter"
}
}
```
Any advice?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.