Aiven-Open / Aiven-Open/http-connector-for-apache-kafka

How can I get the connector to submit the follow curl?

Open
#170 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Java
Stars
119
Forks
63
PR merge metrics
No merged PRs in 30d

Description

There are not enough examples of how to get the http connector to work. Best example I could find is https://redpanda.com/blog/unify-log-data-parseable-redpanda

Here's what I need to submit
```
curl -v --location-trusted -u : -H "strict_mode: true" \
-H "format: json" -H "jsonpaths: [\"$.name\", \"$.code\"]" \
-H "columns: city,tmp_id, id = tmp_id * 100" \
-T example2.json -XPUT \
http://someserver:<443>/api/test_db/table2/_stream_load
```

I need multiple headers to be submitted but the following won't work since http.headers.additional is a String.

```
{
"name": "http-sink-test",
"config": {
"connector.class": "io.aiven.kafka.connect.http.HttpSinkConnector",
"topics.regex": "m1",
"http.authorization.type": "oauth2",
"oauth2.client.id": "username",
"oauth2.client.secret": "password",
"http.headers.additional": [
{
"format": "json"
},
{
"jsonpaths": "[\"$.userid\",\"$.time\",\"$._time\",\"$.remote_user\",\"$.ip\",\"$.request\",\"$.status\",\"$.bytes\",\"$.referrer\",\"$.agent\"]"
}
],
"http.url": "http://https://6dihxzedg.cloud-app.celerdata.com/api/demo/visits/_stream_load",
"batching.enabled": true,
"batch.max.size": 2,
"tasks.max": "1",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter": "org.apache.kafka.connect.json.JsonConverter"
}
}
```

Any advice?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.