indygreg / indygreg/python-zstandard
Provide a file-like object to interface with TarFile
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 642
- Forks
- 116
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 5
Description
I've got a dataset of about 50.000 files. For easier and faster processing, these are inside one tar-archive and compressed with Zstandard. With a file-like object performing decompression, this would be really simple:
```python
with zstd.open('data.tar.zstd', mode='r') as tar:
with tarfile.open(fileobj=tar, mode='r') as archive:
// do something with archive...
```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The requested workflow is shown through zstd.open and tarfile.open(fileobj=tar, mode='r'); start by reading the existing Python decompression entry points and stream interfaces. Done means a decompressed object can be passed to tarfile.open for the demonstrated archive-processing workflow, with the file-like behavior covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100