facebook / facebook/zstd

CLI flag to continuously watch compressed file for changes

Open
#4,426 7 comments 0 reactions 1 assignee Claimed by @Cyan4973 View on GitHub
Dominant language
C
Stars
27.9k
Forks
2.6k
Avg merge
1d 3h
Merged PRs (30d)
8

Description

**Is your feature request related to a problem? Please describe.**
When working in an environment that uses zstd to compress files as they are being written (in my case large logs), having a way to display the content of a compressed log file as it's being written to (updating as data is appended to the file), would be very convenient.
This is however difficult, since zstdless just pipes the output of zstdcat (which obviously does not provide a constant stream of data as the file updates) into less. There is currently no way that I know of to achieve this desired behaviour.

**Describe the solution you'd like**
I forked zstd and changed it to compile an additional (slightly modified `zstdcat`) binary that continuously reads, decompresses, and outputs newly appended data as it's being appended to the compressed input file, not exiting when it reaches EOF.
Using this, I slightly modified the `zstdless` script to pipe this continuous stream of data into `less`, making the behaviour of `zstdless` much more similar to normal `less`.
I would like to see this behaviour implemented in zstd, by the addition of a CLI flag to be passed to zstd when using it for decompression, allowing for an improved implementation of zstdless.
If deemed fitting for the project, I could modify my solution to be incorporated into the project in this way.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.