Nats: Make MaxAckPending and MaxAckWaitTime configurable
- Dominant language
- Go
- Stars
- 2.1k
- Forks
- 274
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 103
Description
When putting heavy load to nats receivers the nats service will possibly redeliver events and therefore duplicate them. This is because we do not configure it correctly.
When pulling events from the queue the expected behaviour is the following:
The event is pulled from the queue and delivered into a channel. When consumed from this channel the event will be acknowledged on the nats server. On high load this can cause problems. Sine `MaxAckWaitTime` is `30s` by default, the nats server will redeliver the event after 30s. Leading to two identical events waiting to be picked up from the channel. Since `MaxAckPending` is set to `1000` by default, this can lead to up to 1000 identical events waiting for being processed.
We already introduced worker groups to be able to handle multiple events at the same time. We now need to make `MaxAckWaitTime` and `MaxAckPending` configurable, so they can configured individually.
Acceptance Criteria:
* update nats-js go-micro package to expose `MaxAckWaitTime` and `MaxAckPending`
* introduce envvars to make them configurable
* set sane defaults
Contributor guide
Assessment
This issue has not been assessed yet.