support compressed data files
Open
@mitzimorris is already working on this.
Since Sep 1, 2021.
enhancement
Feature
i/o
- Dominant language
- C++
- Stars
- 238
- Forks
- 106
- Avg merge
- 2h 29m
- Merged PRs (30d)
- 6
Description
Summary:
Support compressed data files with appropriate suffixes.
Description:
Most data files are highly compressible and compressing cuts down on storage and transport cost.
The behavior should be equivalent to decompressing and then processing, but no intermediate file needs to be created.
Names should follow the usual convention, such as foo.json.gz.
Additional Information:
It looks like Boost has gzip and zip in its iostreams library.
I haven't gotten to the bottom of licensing issues with the zip decompressors. There are several out there.
Current Version:
v2.27.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.