elastic / elastic/logstash

File input stops reading file Logstash 1.5.4 windows

Open
#3,771 4 comments 0 reactions 0 assignees View on GitHub
windows
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

I have Logstash 1.5.4 running on windows 7
I have a syslog server writes to a logfile that logstash points to.
Config file:
# vi logstash-apache.conf

input {
file {
type => "syslogs"
path => "s:/*"
start_position => "beginning"
sincedb_path => "D:/logstash-1.5.4/sinceDB/syslog.db"
}
}
filter
{
if ([message] =~ /^#/) {
drop{}
}
if [type] == "syslogs"
{
grok {
match => { "message" => "%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:monthday} %{TIME:time},%{IP:SourceIP},%{SYSENTRY:Facility},%{SYSENTRY:Severity},%{GREEDYDATA:syslog_message}" }
add_field => [ "datetime", "%{time} %{month} %{monthday} %{year}" ]
}
date {
match => [ "datetime" , "HH:mm:ss MM dd yyyy"]
}
mutate
{
remove_field => [ "month", "monthday","time","year"]
}
}

}
output {
if "_grokparsefailure" in [tags] {
elasticsearch {
host => ES2907
protocol => http
index => "logstash-grokSyslogsfailure-%{+YYYY.MM.dd}"
}
}
else if [type] == "syslogs" {
elasticsearch {
host => ES2907
protocol => http
index => "logstash-syslogs-%{+YYYY.MM.dd}"
}
}

Logstash reads all 247 log files in s:/\* everything is ok, but when it is done it just stops reading the logfile from s:/\* even when the syslog server adds entries to the log file.

If i restart Logstash i picks up the sinceDB and reads the latest entries from the logfile and the stops again. Does it have anything to do with my s: windows mapped drive?

I have turned on the debug log at the last line reads:
{:timestamp=>"2015-08-21T13:43:57.944000+0200", :message=>"_discover_file_glob: s:/*: glob is: [\"s:/150502sys-00.log\", \"s:/150503sys-00.log\", \"s:/150504sys-00.log\", \"s:/150505sys-00.log\", \"s:/150506sys-00.log\", \"s:/150507sys-00.log\", \"s:/150508sys-00.log\", \"s:/150509sys-00.log\", \"s:/150510sys-00.log\", \"s:/150511sys-00.log\", \"s:/150511sys-01.log\", \"s:/150512sys-00.log\", \"s:/150512sys-01.log\", \"s:/150512sys-02.log\", \"s:/150513sys-00.log\", \"s:/150513sys-01.log\", \"s:/150514sys-00.log\", \"s:/150515sys-00.log\", \"s:/150516sys-00.log\", \"s:/150517sys-00.log\", \"s:/150518sys-00.log\", \"s:/150519sys-00.log\", \"s:/150519sys-01.log\", \"s:/150520sys-00.log\", \"s:/150520sys-01.log\", \"s:/150520sys-02.log\", \"s:/150521sys-00.log\", \"s:/150521sys-01.log\", \"s:/150522sys-00.log\", \"s:/150523sys-00.log\", \"s:/150524sys-00.log\", \"s:/150525sys-00.log\", \"s:/150526sys-00.log\", \"s:/150527sys-00.log\", \"s:/150528sys-00.log\", \"s:/150528sys-01.log\", \"s:/150529sys-00.log\", \"s:/150530sys-00.log\", \"s:/150531sys-00.log\", \"s:/150601sys-00.log\", \"s:/150602sys-00.log\", \"s:/150603sys-00.log\", \"s:/150604sys-00.log\", \"s:/150605sys-00.log\", \"s:/150606sys-00.log\", \"s:/150607sys-00.log\", \"s:/150608sys-00.log\", \"s:/150608sys-01.log\", \"s:/150609sys-00.log\", \"s:/150610sys-00.log\", \"s:/150611sys-00.log\", \"s:/150612sys-00.log\", \"s:/150612sys-01.log\", \"s:/150613sys-00.log\", \"s:/150614sys-00.log\", \"s:/150615sys-00.log\", \"s:/150616sys-00.log\", \"s:/150617sys-00.log\", \"s:/150618sys-00.log\", \"s:/150619sys-00.log\", \"s:/150620sys-00.log\", \"s:/150621sys-00.log\", \"s:/150622sys-00.log\", \"s:/150623sys-00.log\", \"s:/150624sys-00.log\", \"s:/150625sys-00.log\", \"s:/150626sys-00.log\", \"s:/150627sys-00.log\", \"s:/150628sys-00.log\", \"s:/150629sys-00.log\", \"s:/150630sys-00.log\", \"s:/150701sys-00.log\", \"s:/150702sys-00.log\", \"s:/150703sys-00.log\", \"s:/150704sys-00.log\", \"s:/150705sys-00.log\", \"s:/150706sys-00.log\", \"s:/150707sys-00.log\", \"s:/150708sys-00.log\", \"s:/150709sys-00.log\", \"s:/150710sys-00.log\", \"s:/150711sys-00.log\", \"s:/150712sys-00.log\", \"s:/150713sys-00.log\", \"s:/150714sys-00.log\", \"s:/150715sys-00.log\", \"s:/150716sys-00.log\", \"s:/150717sys-00.log\", \"s:/150718sys-00.log\", \"s:/150719sys-00.log\", \"s:/150720sys-00.log\", \"s:/150721sys-00.log\", \"s:/150722sys-00.log\", \"s:/150723sys-00.log\", \"s:/150724sys-00.log\", \"s:/150725sys-00.log\", \"s:/150726sys-00.log\", \"s:/150727sys-00.log\", \"s:/150728sys-00.log\", \"s:/150728sys11-00.log\", \"s:/150728sys12-00.log\", \"s:/150729sys-00.log\", \"s:/150730sys-00.log\", \"s:/150731sys-00.log\", \"s:/150801sys-00.log\", \"s:/150802sys-00.log\", \"s:/150803sys-00.log\", \"s:/150804sys-00.log\", \"s:/150805sys-00.log\", \"s:/150806sys-00.log\", \"s:/150807sys-00.log\", \"s:/150808sys-00.log\", \"s:/150809sys-00.log\", \"s:/150810sys-00.log\", \"s:/150811sys-00.log\", \"s:/150812sys-00.log\", \"s:/150813sys-00.log\", \"s:/150814sys-00.log\", \"s:/150815sys-00.log\", \"s:/150816sys-00.log\", \"s:/150817sys-00.log\", \"s:/150818sys-00.log\", \"s:/150819sys-00.log\", \"s:/150820sys-00.log\", \"s:/150821sys-00.log\", \"s:/2015-01-01.csv\", \"s:/2015-01-02.csv\", \"s:/2015-01-03.csv\", \"s:/2015-01-04.csv\", \"s:/2015-01-05.csv\", \"s:/2015-01-06.csv\", \"s:/2015-01-07.csv\", \"s:/2015-01-08.csv\", \"s:/2015-01-09.csv\", \"s:/2015-01-10.csv\", \"s:/2015-01-11.csv\", \"s:/2015-01-12.csv\", \"s:/2015-01-13.csv\", \"s:/2015-01-14.csv\", \"s:/2015-01-15.csv\", \"s:/2015-01-16.csv\", \"s:/2015-01-17.csv\", \"s:/2015-01-18.csv\", \"s:/2015-01-19.csv\", \"s:/2015-01-20.csv\", \"s:/2015-01-21.csv\", \"s:/2015-01-22.csv\", \"s:/2015-01-23.csv\", \"s:/2015-01-24.csv\", \"s:/2015-01-25.csv\", \"s:/2015-01-26.csv\", \"s:/2015-01-27.csv\", \"s:/2015-01-28.csv\", \"s:/2015-01-29.csv\", \"s:/2015-01-30.csv\", \"s:/2015-01-31.csv\", \"s:/2015-02-01.csv\", \"s:/2015-02-02.csv\", \"s:/2015-02-03.csv\", \"s:/2015-02-04.csv\", \"s:/2015-02-05.csv\", \"s:/2015-02-06.csv\", \"s:/2015-02-07.csv\", \"s:/2015-02-08.csv\", \"s:/2015-02-09.csv\", \"s:/2015-02-10.csv\", \"s:/2015-02-11.csv\", \"s:/2015-02-12.csv\", \"s:/2015-02-13.csv\", \"s:/2015-02-14.csv\", \"s:/2015-02-15.csv\", \"s:/2015-02-16.csv\", \"s:/2015-02-17.csv\", \"s:/2015-02-18.csv\", \"s:/2015-02-19.csv\", \"s:/2015-02-20.csv\", \"s:/2015-02-21.csv\", \"s:/2015-02-22.csv\", \"s:/2015-02-23.csv\", \"s:/2015-02-24.csv\", \"s:/2015-02-25.csv\", \"s:/2015-02-26.csv\", \"s:/2015-02-27.csv\", \"s:/2015-02-28.csv\", \"s:/2015-03-01.csv\", \"s:/2015-03-02.csv\", \"s:/2015-03-03.csv\", \"s:/2015-03-04.csv\", \"s:/2015-03-05.csv\", \"s:/2015-03-06.csv\", \"s:/2015-03-07.csv\", \"s:/2015-03-08.csv\", \"s:/2015-03-09.csv\", \"s:/2015-03-10.csv\", \"s:/2015-03-11.csv\", \"s:/2015-03-12.csv\", \"s:/2015-03-13.csv\", \"s:/2015-03-14.csv\", \"s:/2015-03-15.csv\", \"s:/2015-03-16.csv\", \"s:/2015-03-17.csv\", \"s:/2015-03-18.csv\", \"s:/2015-03-19.csv\", \"s:/2015-03-20.csv\", \"s:/2015-03-21.csv\", \"s:/2015-03-22.csv\", \"s:/2015-03-23.csv\", \"s:/2015-03-24.csv\", \"s:/2015-03-25.csv\", \"s:/2015-03-26.csv\", \"s:/2015-03-27.csv\", \"s:/2015-03-28.csv\", \"s:/2015-03-29.csv\", \"s:/2015-03-30.csv\", \"s:/2015-03-31.csv\", \"s:/2015-04-01.csv\", \"s:/2015-04-02.csv\", \"s:/2015-04-03.csv\", \"s:/2015-04-04.csv\", \"s:/2015-04-05.csv\", \"s:/2015-04-06.csv\", \"s:/2015-04-07.csv\", \"s:/2015-04-08.csv\", \"s:/2015-04-09.csv\", \"s:/2015-04-10.csv\", \"s:/2015-04-11.csv\", \"s:/2015-04-12.csv\", \"s:/2015-04-13.csv\", \"s:/2015-04-14.csv\", \"s:/2015-04-15.csv\", \"s:/2015-04-16.csv\", \"s:/2015-04-17.csv\", \"s:/2015-04-18.csv\", \"s:/2015-04-19.csv\", \"s:/2015-04-20.csv\", \"s:/2015-04-21.csv\", \"s:/2015-04-22.csv\", \"s:/2015-04-23.csv\", \"s:/2015-04-24.csv\", \"s:/2015-04-25.csv\", \"s:/2015-04-26.csv\", \"s:/2015-04-27.csv\", \"s:/2015-04-28.csv\", \"s:/2015-04-29.csv\", \"s:/2015-04-30.csv\", \"s:/2015-05-01.csv\", \"s:/2015-05-02.csv\"]", :level=>:debug, :file=>"/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/filewatch-0.6.5/lib/filewatch/watch.rb", :line=>"132", :method=>"_discover_file"}

}

Best regards

Kåre

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.