influxdata / influxdata/telegraf

tail plugin issue

Open
#13,580 9 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Relevant telegraf.conf

```toml
[global_tags]
job = "yibo-in-out"

[[inputs.tail]]
files = ["D:\\LPN5.07\\SystemLog\\*.log"]
# Read file from beginning.
from_beginning = false
data_format = "grok"
grok_patterns = ['^.*\[INFO\].*上报请求信息%{GREEDYDATA:message}$']

[[inputs.cpu]]
## Whether to report per-cpu stats or not
percpu = true
## Whether to report total system cpu stats or not
totalcpu = true
## If true, collect raw CPU time metrics
collect_cpu_time = false
## If true, compute and report the sum of all non-idle CPU states
report_active = false
## If true and the info is available then add core_id and physical_id tags
core_tags = false

[agent]
## Default data collection interval for all inputs
interval = "10s"
## Rounds collection interval to 'interval'
## ie, if interval="10s" then always collect on :00, :10, :20, etc.
round_interval = true
debug = true
logtarget = "file"
logfile = "C:\\Program Files\\telegraf\\logs.log"
logfile_rotation_max_size = "10MB"
logfile_rotation_max_archives = 7
log_with_timezone = "Asia/Shanghai"

## Telegraf will send metrics to outputs in batches of at most
## metric_batch_size metrics.
## This controls the size of writes that Telegraf sends to output plugins.
metric_batch_size = 10

[[processors.rename]]
## Specify one sub-table per rename operation.
[[processors.rename.replace]]
measurement = "tail"
dest = "parking-in-out"

[[processors.parser]]
parse_fields = ["message"]
drop_original = true
json_string_fields = ["VehicleNo","ChannelName"]
json_time_key = "EventTime"
json_time_format = "2006-01-02 15:04:05"
data_format = "json"
json_timezone = "Asia/Shanghai"

[[outputs.influxdb_v2]]
urls = ["http://10.200.101.17:8086"]
## Token for authentication.
token = "xxxxxxx"
organization = "icep"
bucket = "yiboinfo"
```

### Logs from Telegraf

```text
2023-07-07T18:03:01+08:00 D! [parsers.grok::tail] Grok no match found for: "[2023:07:07 18:02:58][DEBUG]出口通道号16,发送系统时间2023年07月07日"
2023-07-07T18:03:01+08:00 D! [parsers.grok::tail] Grok no match found for: "[2023:07:07 18:02:58][ERROR]通道号[2],IP[172.16.60.11]登陆成功后不能通讯"
2023-07-07T18:03:06+08:00 D! [outputs.influxdb_v2] Wrote batch of 4 metrics in 1.0002ms
2023-07-07T18:03:06+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:16+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:26+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:36+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:46+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2023-07-07T18:03:56+08:00 D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
```

### System info

telegraf 1.27, Windows server 2012 R2 Datacenter

### Docker

_No response_

### Steps to reproduce

1. set telegraf.conf file, and start. telegraf grok record and send out to influxdb.
2. in the beginning, working file, telegraf can get record, but hours, telegraf can not get any record. although new message coming into logs file.

### Expected behavior

telegraf can analysis new message in logs file continually,

### Actual behavior

telegraf is running ( from telegraf log, seemed telegraf working fine, no any issue report), but telegraf can not get new message and grok.

new message still coming from log file in `D:\\LPN5.07\\SystemLog`
```
2023:07:07 18:02:58][DEBUG]出口通道号16,发送系统时间2023年07月07日
[2023:07:07 18:02:58][INFO]出场场上报请求信息{"ParkCode":"510105037","VehicleNo":"GJ9L91","EventTime":"2023-07-07 18:02:20","StartTime":"2023-07-07 12:59:39","PaymentMoney":14.0,"TotalMoney":14.0,"DeductionMoney":0.0,"IsAutoPay":false,"CouponName":"","CouponMoney":0.0,"OperateCount":1738,"Qn":"2307071802200000008","BookingCode":"","ConsumeType":1,"PayType":2,"ChannelNo":8,"ChannelName":"3号楼出口2","CarImageID":"2307071802200000008","ISETCPay":false,"ETCPaymentMoney":0.0}
[2023:07:07 18:02:58][ERROR]通道号[2],IP[172.16.60.11]登陆成功后不能通讯
[2023:07:07 18:02:58][ERROR]通道号[4],IP[172.16.60.13]登陆成功后不能通讯
[2023:07:07 18:02:58][ERROR]通道号[1],IP[172.16.60.10]一体机重连不能通讯
[2023:07:07 18:02:58][ERROR]通道号[13],IP[10.18.212.12]一体机重连不能通讯
[2023:07:07 18:02:58][ERROR]通道号[11],IP[10.18.212.10]一体机重连不能通讯
[2023:07:07 18:02:58][ERROR]通道号[3],IP[172.16.60.12]一体机重连不能通讯
[2023:07:07 18:03:00][ERROR]通道号[2],IP[172.16.60.11]登陆成功后不能通讯
```

### Additional info

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported behavior with the `inputs.tail` plugin and its Grok parser using the supplied telegraf.conf, logs, and Windows paths. Trace why new log lines stop being collected after several hours, then verify that sustained input continues to produce metrics and that the existing Grok matches still work.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.