Periodically read content of a file using `file` source

As a user, I want to periodically read entire content of a given file (or all files matching a glob pattern) and produce events with this content as the message field. The main use case is reading system observability data from procfs and sysfs on Linux.

Proposal

One idea to implement this is to introduce new fingerprinting strategy called none, which would not fingerprint files at all and thus re-read them every glob_minimum_cooldown milliseconds.

Examples

If I wanted to monitor CPU temperature with 5 seconds interval, the config could have looked like this:

[inputs.cpu_temp]
type = "file"
include = ["/sys/devices/virtual/thermal/thermal_zone0/temp"]
glob_minimum_cooldown =  5000 # default 1000
fingerprinting.strategy = "none"

or, if I wanted to log cumulative load of all CPUs since start of the system (see section about /proc/stat in the manpage), like

[inputs.cpu_load_read]
type = "file"
include = ["/proc/stat"]
message_start_indicator = "^cpu\\s"
fingerprinting.strategy = "none"

[inputs.cpu_load_extract]
type = "regex_parser"
inputs = ["cpu_load_read"]
regex = "^cpu\\s+(?P<user>\\d+) (?P<nice>\\d+) (?P<system>\\d+) (?P<idle>\\d+)"

Caveats

There is a minor issue with this approach: if files are not fingerprinted, on each read there would be messages in Vector's log looking like

Nov 17 12:50:04.236  INFO source{name=cpu_temp}:file_server: file_source::file_server: Found file to watch. path="/sys/devices/virtual/thermal/thermal_zone0/temp" file_position=0

which means that the file names has to be remembered by Vector to avoid noise in the log. Alternatively, we can just stop logging names of found files at all for this fingerprinting strategy.

Related

Use cases of this with procfs and sysfs are somewhat similar to the ones of #992.

Start by reading the existing file source behavior around fingerprinting, glob_minimum_cooldown, and message assembly. Compare the proposal’s fingerprinting.strategy = "none" examples for /sys and /proc with current behavior. Done means matching files are reread periodically as complete message events without repeated found-file log noise.