elastic / elastic/beats

decompress_gzip_field Failed to decompress field in decompress_gzip_field processor: error decompressing field message: gzip: invalid header

Open
#40,037 13 comments 0 reactions 0 assignees View on GitHub
enhancement Team:Elastic-Agent-Data-Plane
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 1h
Merged PRs (30d)
370

Description

I am trying to decompress a message with the https://www.elastic.co/guide/en/beats/filebeat/current/decompress-gzip-field.html
but it seems that it does not work. I get `error decompressing field gzip: invalid header`

The compression used for this field is unfortunately zlib forced by the following python lib:
https://github.com/keeprocking/pygelf/blob/master/pygelf/gelf.py#L85

and the decompress-gzip-field seems to only accept gzip
https://github.com/elastic/beats/blob/main/libbeat/processors/actions/decompress_gzip_field.go#L108

Please see the history here:
https://stackoverflow.com/questions/20762094/how-are-zlib-gzip-and-zip-related-what-do-they-have-in-common-and-how-are-they

would be nice for decompress_gzip_field processor to accept zlib as well
https://pkg.go.dev/compress/zlib

Love you

Contributor guide

Open the contributing guide

Research direction

Start by reading libbeat/processors/actions/decompress_gzip_field.go at the referenced line and compare its gzip handling with Go's compress/zlib package. Done means the decompress_gzip_field processor accepts the zlib-compressed field produced by pygelf while preserving existing gzip behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, python
Domain
backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.