elastic / elastic/logstash

[sincedb file is not updated properly]

Open
#4,221 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

I am using logstash to parse files in folder .what I see in the terminal after some time a message is coming Logstash startup completed

io/console not supported; tty will not be manipulated
Default settings used: Filter workers: 4
Logstash startup completed.

There are three sincedb files created and each contains info for one file which is as follows

370201013-11863-65536 0 3 246

But ideally in each file there should be info for 10 files .When i press control ^c i got following message on screen
^C←[33mSIGINT received. Shutting down the pipeline. {:level=>:warn}←[0mTerminate
batch job (Y/N)?
Logstash shutdown completed

After that the since db files contains the info for all the 10 files which is as follows

370201013-11851-65536 0 3 246
370201013-11863-65536 0 3 246
370201013-11869-65536 0 3 246
370201013-11899-65536 0 3 246
370201013-11881-65536 0 3 246
370201013-11857-65536 0 3 246
370201013-11887-65536 0 3 246
370201013-11905-65536 0 3 246
370201013-11893-65536 0 3 246
370201013-11875-65536 0 3 246

Can you tell me how can get info for all the files in since db at the first instance .

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the folder-parsing setup and inspect how the sincedb files change between Logstash startup and shutdown. Compare the records present during processing with those written after Ctrl-C, then determine the expected first-instance behavior and document whether the reported discrepancy is a persistence bug.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.