influxdata / influxdata/telegraf
Error starting telegraf for azure eventhub_consumer with persistence file
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
I have successfully created a telegraf stream for device data from azure iot hub, using the eventhub_consumer.
However, to not re-reading and consuming again all (millions) of messages in the 7d buffer of the stream, I opted in using file persistence and not only in memory persistence - as per documentation.
After careful configuration and multiple tests and code review there is a bug.
I get....
`E! [telegraf] Error running agent: starting input inputs.eventhub_consumer: creating receiver for partition "1": open [FILEPATH]/[IOTHUB-NAME]-24996945-d1f3232547_hanp1iottest_influxdb_0: no such file or directory`
..in the log. (tested with multiple file locations and permissions)
The problem is, the persister does not create inital files for persistance, and then can't open them. Not sure where in the code the bug is, if it could be a result of some sort of a race condition or not.
**Found a "silly" Workaround**: Created the mentioned files in the log, withing the directory, with the content `{}` (empty JSON) and it seems to work as expected now, the files are persisting the state between restarts. This could easily break when adding new partitions in the eventhub, so you need to add new files as you add partitions.
Using latest telegraf agent in ubuntu 22. Telegraf 1.26.3 (git: HEAD@90f4eb29) @ Ubuntu 22.04.2 LTS
Contributor guide
Research direction
Start at the eventhub_consumer input and its file-persistence receiver setup, then reproduce the failure using the configuration described in the issue. Check startup with a persistence directory that has no partition files. Done means Telegraf starts without manually creating empty JSON files, preserves state across restarts, and handles newly added partitions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, go
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100