allenai / allenai/cached_path

Add support for zstandard, bzip2, and lzma compressed tar files

未关闭
#259 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
Feature request
主要语言
Python
星标
47
派生
22
PR 合并指标
30 天内没有已合并 PR

描述

**Is your feature request related to a problem? Please describe.**
Currently this library only supports `*.tar.gz` (gzip) files. This needs to be extended to also support `*.tar.zst` (zstandard), `*.tar.bz2` (bzip2), and `*.tzr.xz` (lzma) files

**Describe the solution you'd like**
Automatically decompress the compressed tar-file using the appropriate decompressor. In Python 3.14+ this can be accomplished using `mode` kwarg in `tarfile.open` ([link](https://docs.python.org/3.14/library/tarfile.html#tarfile.open)). In older Python versions, we will need special handling for zstandard compressed tar files (using the standalone [python-zstandard](https://github.com/indygreg/python-zstandard) library) since `tarfile.open` doesn't yet support zstandard.

**Describe alternatives you've considered**
None

**Additional context**
None

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。