influxdata / influxdata/telegraf
tail plugin issue
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
### Relevant telegraf.conf
```toml
[global_tags]
job = "yibo-in-out"
[[inputs.tail]]
files = ["D:\\LPN5.07\\SystemLog\\*.log"]
# Read file from beginning.
from_beginning = false
data_format = "grok"
grok_patterns = ['^.*\[INFO\].*上报请求信息%{GREEDYDATA:message}$']
[[inputs.cpu]]
## Whether to report per-cpu stats or not
percpu = true
## Whether to report total system cpu stats or not
totalcpu = true
## If true, collect raw CPU time metrics
collect_cpu_time = false
## If true, compute and report the sum of all non-idle CPU states
report_active = false
## If true and the info is available then add core_id and physical_id tags
core_tags = false
[agent]
## Default data collection interval for all inputs
interval = "10s"
## Rounds collection interval to 'interval'
## ie, if interval="10s" then always collect on :00, :10, :20, etc.
round_interval = true
debug = true
logtarget = "file"
logfile = "C:\\Program Files\\telegraf\\logs.log"
logfile_rotation_max_size = "10MB"
logfile_rotation_max_archives = 7
log_with_timezone = "Asia/Shanghai"
## Telegraf will send metrics to outputs in batches of at most
## metric_batch_size metrics.
## This controls the size of writes that Telegraf sends to output plugins.
metric_batch_size = 10
[[processors.rename]]
## Specify one sub-table per rename operation.
[[processors.rename.replace]]
measurement = "tail"
dest = "parking-in-out"
[[processors.parser]]
parse_fields = ["message"]
drop_original = true
json_string_fields = ["VehicleNo","ChannelName"]
json_time_key = "EventTime"
json_time_format = "2006-01-02 15:04:05"
data_format = "json"
json_timezone = "Asia/Shanghai"
[[outputs.influxdb_v2]]
urls = ["http://10.200.101.17:8086"]
## Token for authentication.
token = "xxxxxxx"
organization = "icep"
bucket = "yiboinfo"
```
### Logs from Telegraf
```text
2023-07-07T18:03:01+08:00 D! [parsers.grok::tail] Grok no match found for: "[2023:07:07 18:02:58][DEBUG]出口通道号16,发送系统时间2023年07月07日"
2023-07-07T18:03:01+08:00 D! [parsers.grok::tail] Grok no match found for: "[2023:07:07 18:02:58][ERROR]通道号[2],IP[172.16.60.11]登陆成功后不能通讯"
2023-07-07T18:03:06+08:00 D! [outputs.influxdb_v2] Wrote batch of 4 metrics in 1.0002ms
2023-07-07T18:03:06+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:16+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:26+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:36+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:46+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:56+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
```
### System info
telegraf 1.27, Windows server 2012 R2 Datacenter
### Docker
_No response_
### Steps to reproduce
1. set telegraf.conf file, and start. telegraf grok record and send out to influxdb.
2. in the beginning, working file, telegraf can get record, but hours, telegraf can not get any record. although new message coming into logs file.
### Expected behavior
telegraf can analysis new message in logs file continually,
### Actual behavior
telegraf is running ( from telegraf log, seemed telegraf working fine, no any issue report), but telegraf can not get new message and grok.
new message still coming from log file in `D:\\LPN5.07\\SystemLog`
```
2023:07:07 18:02:58][DEBUG]出口通道号16,发送系统时间2023年07月07日
[2023:07:07 18:02:58][INFO]出场场上报请求信息{"ParkCode":"510105037","VehicleNo":"GJ9L91","EventTime":"2023-07-07 18:02:20","StartTime":"2023-07-07 12:59:39","PaymentMoney":14.0,"TotalMoney":14.0,"DeductionMoney":0.0,"IsAutoPay":false,"CouponName":"","CouponMoney":0.0,"OperateCount":1738,"Qn":"2307071802200000008","BookingCode":"","ConsumeType":1,"PayType":2,"ChannelNo":8,"ChannelName":"3号楼出口2","CarImageID":"2307071802200000008","ISETCPay":false,"ETCPaymentMoney":0.0}
[2023:07:07 18:02:58][ERROR]通道号[2],IP[172.16.60.11]登陆成功后不能通讯
[2023:07:07 18:02:58][ERROR]通道号[4],IP[172.16.60.13]登陆成功后不能通讯
[2023:07:07 18:02:58][ERROR]通道号[1],IP[172.16.60.10]一体机重连不能通讯
[2023:07:07 18:02:58][ERROR]通道号[13],IP[10.18.212.12]一体机重连不能通讯
[2023:07:07 18:02:58][ERROR]通道号[11],IP[10.18.212.10]一体机重连不能通讯
[2023:07:07 18:02:58][ERROR]通道号[3],IP[172.16.60.12]一体机重连不能通讯
[2023:07:07 18:03:00][ERROR]通道号[2],IP[172.16.60.11]登陆成功后不能通讯
```
### Additional info
_No response_
Contributor guide
Research direction
Start by reproducing the reported behavior with the `inputs.tail` plugin and its Grok parser using the supplied telegraf.conf, logs, and Windows paths. Trace why new log lines stop being collected after several hours, then verify that sustained input continues to produce metrics and that the existing Grok matches still work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100