Environment variable for --memory
- Dominant language
- C
- Stars
- 27.9k
- Forks
- 2.6k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 8
Description
**Is your feature request related to a problem? Please describe.**
Sometimes I use tools that utilize zstd, but they don't allow me to pass arbitrary flags to zstd (in this case `rg -z`). In these cases, the tool won't work in cases where the archive requires additional flags to decompress (such as needing `--long=32` or `--memory=2048MB`)
**Describe the solution you'd like**
It would be great if there was an environment variable like ZSTD_MEMORY or similar to specify window size for compression/decompression.
This would allow me to use tools where I'm not able to pass options to zstd to decompress files that would need a flag like `--long=32` or `--memory=2048MB`.
**Describe alternatives you've considered**
In this case, I wrote a shell script using `xargs` that ran `zstdcat --long=31` on files and piped them to `rg`.
**Additional context**
In this case, I was using `rg` to gather statistics on a large compressed text corpora. Usually `rg` is nice for search a lot of compressed files in a directory, but these had been compressed with --long, so I got an error[1].
I also [opened a bug](https://github.com/BurntSushi/ripgrep/issues/2781) in the ripgrep repo to support passing flags to the compression libraries it uses. That said, this would be a really nice general feature of zstd.
Thank you! :)
**Footnotes**
[1] Error message from rg, bubbling up an error from zstd:
```
-------------------------------------------------------------------------------
./submissions/RS_2018-01.zst:
-------------------------------------------------------------------------------
sions/RS_2018-01.zst : Decoding error (36) : Frame requires too much memory for decoding
sions/RS_2018-01.zst : Window size larger than maximum : 2147483648 > 134217728
sions/RS_2018-01.zst : Use --long=31 or --memory=2048MB
-------------------------------------------------------------------------------
```
Contributor guide
Assessment
This issue has not been assessed yet.