opensearch-project / opensearch-project/data-prepper

[BUG] Kafka Producer Callback releases events incorrectly

Open
#3,680 0 comments 0 reactions 1 assignee View on GitHub

@kkondaka is already working on this.

Since Nov 16, 2023.

bug
Dominant language
Java
Stars
374
Forks
355
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug
The following code - has a bug about releasing event handles without trying to push the events to DLQ first.

https://github.com/opensearch-project/data-prepper/blob/main/data-prepper-plugins/kafka-plugins/src/main/java/org/opensearch/dataprepper/plugins/kafka/producer/KafkaCustomProducer.java#L198-L210

It should be fixed such a way that - if DLQ is not configured or DLQ push fails, events should be released with negative ack, otherwise they should be released with positive ack

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: [e.g. Ubuntu 20.04 LTS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.