How to parse .gz (compressed file) using logstash

Hi All,

Does latest logstash support parsing .gz files using logstash?

We have a requirement to get .gz files dynamically every hour. We do not know the file names and content. All we know is the file is compressed in .gz format.

We are using logstash version 5.6.0 and using gzip_lines codec plugin but logstash is not able to parse .gz files. Also using grok filter for parsing/extracting fields.

Sample code

input {
file {
type => "test-xyz"
path => "/tmp/*.gz"
start_position => "beginning"
sincedb_path => "/dev/null"
codec => "gzip_lines"
}
}

I have seen some old threads saying logstash does not support .gz files.
Wanted to check whether there is any work around for dynamic .gz files we receive.

Please provide your inputs on this.

Thanks and Regards,
Jotsna

The issue provides a file input configuration using /tmp/*.gz, start_position, sincedb_path, and the gzip_lines codec. Begin by reproducing this setup with Logstash 5.6.0 and checking the file input and codec behavior; done means establishing whether dynamic .gz ingestion is supported and documenting the supported path or limitation.

Written by the indexing model from the issue text.