Aiven-Open / Aiven-Open/http-connector-for-apache-kafka
Question: Retry logic/configuration for Connector
- 主要语言
- Java
- 星标
- 119
- 派生
- 63
- PR 合并指标
- 30 天内没有已合并 PR
描述
I have a use case where I want the HTTP Connector to retry delivering failed messages. That part, I've figured out and have it configured to my liking.
The thing is, in my case, I want the HTTP Connector to retry delivering the message several times before giving up **on that message** - then evict/discard that message and move onto the next message in the Kafka Topic queue. Right now, after the error threshold, the Aiven HTTP Connector fails outright, and the Aiven HTTP Connector does not deliver any further messages, holding up the entire queue. Below is my configuration:
```
{
"name": "EXAMPLE_MESSAGE",
"config": {
"connector.class": "io.aiven.kafka.connect.http.HttpSinkConnector",
"topics.regex": "EXAMPLE_TOPIC",
"http.authorization.type": "none",
"http.url": "https://api.domain.local/path/to/my/api",
"batching.enabled": "false",
"batch.max.size": 1,
"tasks.max": "1",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "false",
"group.id": "my_group_name",
"retry.backoff.ms": 3000,
"max.retries": 5,
"consumer.override.auto.offset.reset": "latest"
}
}
```
How do I achieve the outcome I want? Where if message `#1` fails after several attempts, the Aiven HTTP Connector will just move onto message `#2`.
贡献指南
这个仓库没有索引到贡献指南
评估
这个 Issue 还没有评估数据。