allenai / allenai/cached_path

Add support for zstandard, bzip2, and lzma compressed tar files

Aperta
#259 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Feature request
Lingua principale
Python
Stelle
47
Fork
22
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

**Is your feature request related to a problem? Please describe.**
Currently this library only supports `*.tar.gz` (gzip) files. This needs to be extended to also support `*.tar.zst` (zstandard), `*.tar.bz2` (bzip2), and `*.tzr.xz` (lzma) files

**Describe the solution you'd like**
Automatically decompress the compressed tar-file using the appropriate decompressor. In Python 3.14+ this can be accomplished using `mode` kwarg in `tarfile.open` ([link](https://docs.python.org/3.14/library/tarfile.html#tarfile.open)). In older Python versions, we will need special handling for zstandard compressed tar files (using the standalone [python-zstandard](https://github.com/indygreg/python-zstandard) library) since `tarfile.open` doesn't yet support zstandard.

**Describe alternatives you've considered**
None

**Additional context**
None

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.