influxdata / influxdata/kapacitor
ServerID is not accounted when consuming subscription
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
We run Kapacitor as a Docker container (`FROM kapacitor:1.5.1`). And as we see no need to persist any "runtime" data, we don't use any dedicated volume for it.
However, what we sometimes do is change Kapacitor configuration (`conf`, handlers, alert definitions), which is then followed by image re-build and container re-run, while Kapacitor URL/port remains the same.
We also have some subscriptions configured in our `kapacitor.conf`:
```
[[influxdb]]
...
[influxdb.subscriptions]
telegraf = ["autogen"]
...
```
Which causes Kapacitor to create a new subscription every time the server is initialized, if subscription is not there. And it uses the current Kapacitor instance' `ServerID` to name a subscription. So on our InfluxDB (`1.4.2`) server it then looks like:
```
> show subscriptions
name: telegraf
retention_policy name mode destinations
---------------- ---- ---- ------------
autogen kapacitor-c11618fc-5af5-4a1c-bb81-a66d19f0fa23 ANY [http://my.kapacitor.hostname:9092]
```
The problem is that the mentioned `ServerID` attribute changes every time the server is initialized (container is re-run), which makes it to create a new subscription as this time the name for a subscription would not have a match within the existing subscriptions. So we end up with a new subscription added, something like:
```
> show subscriptions
name: telegraf
retention_policy name mode destinations
---------------- ---- ---- ------------
autogen kapacitor-c11618fc-5af5-4a1c-bb81-a66d19f0fa23 ANY [http://my.kapacitor.hostname:9092]
autogen kapacitor-4399ee14-a4aa-4c11-9d55-1f3c99ef6223 ANY [http://my.kapacitor.hostname:9092]
```
That wouldn't be a problem, considering that Kapacitor would accept a subscription data only if URL matches and name matches (something I derived from https://github.com/influxdata/kapacitor/issues/349#issuecomment-197908046). But the problem is that with 2 existing subscriptions as above, I then receive a duplicated data when using a `stream()` node - each coming from the existing subscription. And if I drop one subscription, I stop getting the duplicated data. If I instead re-run the Kapacitor again it'll generate a new subscription and I'll be receiving the data thrice, and so on. So it looks like a subscription name (which includes `ServerID`) is not accounted when Kapacitor decides whether to accept a data or not.
What would be a suggested solution to run Kapacitor in such environment and scenario?
I can create a container pre-run script which wipes out not used subscriptions, but it looks more like a workaround. Alternatively, I can remove subscription section in `kapacitor.conf` and pre-create a subscription manually - but this is a workaround too. Even if Kapacitor would account the name (comprised of `ServerID`) of a subscription, I would still end up with tens of not used subscriptions accumulated over the time (considering we re-deploy the container often) - what would be a solution for Kapacitor then?
Contributor guide
Research direction
No source files, tests, or entry points are named in the report. Start by locating Kapacitor's subscription creation and subscription-data matching paths, then reproduce repeated container initialization against InfluxDB; done would require an agreed solution that prevents duplicate delivery and handles obsolete subscriptions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100