meta-pytorch / meta-pytorch/data
Add zlib support to Decompressor
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
🚀 The feature
Add support to Decompressor for file_type="zlib", and add a corresponding ZlibFileLoader.
Motivation, pitch
zlib is a very popular format, but Decompressor doesn't seem to support it (AFAICT it only supports gzip, lzma, tar, zip, and bzip2). I ran into this while working with a large dataset compressed using zlib and had to roll my own solution.
Alternatives
No response
Additional context
I'm happy to submit a PR implementing this.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the Decompressor entry point and the existing file loaders, then compare how their supported file types are wired in. Add support for file_type="zlib" and a corresponding ZlibFileLoader; done means zlib-compressed datasets can use the same loading path as the formats already supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100