archive_util bypasses tarfile's PEP 706 extraction filters by calling private APIs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 1.4k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 1
Description
setuptools.archive_util.unpack_tarfile extracts through three private tarfile APIs, which means none of the hardening CPython added in PEP 706 ever runs.
# unpack_tarfile
tarobj._extract_member(member, final_dst)
# _iter_open_tar
tar_obj.chown = lambda *args: None
# _resolve_tar_file_or_dir
tar_member_obj = tar_obj._getmember(linkpath)
Why it matters
_extract_member is the internal worker that extractall calls after the extraction filter has vetted and rewritten the member. Calling it directly skips the filter entirely, so setuptools gets none of data_filter's protections: absolute and upward link targets, device and FIFO members, setuid/setgid/sticky bits, and permission handling. tar_obj.chown = lambda *args: None is a hand-rolled substitute for one small piece of what the filter already does properly.
The containment check added in #5325 re-implements one of those protections by hand. That was the right scope for a security fix, but it means we now maintain our own partial copy of logic the stdlib maintains fully, and the remaining gaps stay open.
Being private API is a liability in its own right: signature or behavior can change in any release with no deprecation, and _getmember in particular has no stability guarantee.
The complication
unpack_tarfile cannot simply become tarobj.extractall(extract_dir, filter='data'). The documented progress_filter callback takes (source_path, dest_path) and may return a different destination or None to skip, so callers can redirect individual members anywhere they like — including, legitimately, outside the nominal extraction directory. That is why #5325 applies its containment check to the preliminary destination, before progress_filter runs.
Reconciling the two is the actual work here. Rough options:
- Run each member through
tarfile.data_filter(member, extract_dir)for validation and to pick up its sanitizedTarInfo, then extract to the possibly-redirected destination. Keepsprogress_filtersemantics; still uses_extract_memberfor the write. - Extract to a temporary directory with
extractall(filter='data'), then move members into place according toprogress_filter. Fully public API, at the cost of an extra copy and of changing when the callback is invoked relative to extraction. - Reduce the
progress_filtercontract to filtering only, not redirection — a breaking change, but it would let the stdlib own extraction entirely.
tarfile.data_filter is available on all currently supported Pythons (requires-python = ">=3.10") via the 3.10.12 / 3.11.4 security backports, though the exact minimum patch level is worth confirming before relying on it unconditionally.
Split out of #5325, where this was noted but deliberately left alone as out of scope for the security fix.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with setuptools.archive_util.unpack_tarfile, _iter_open_tar, and _resolve_tar_file_or_dir, then inspect tarfile.data_filter and the changes from #5325. Confirm the minimum supported Python patch levels before choosing an approach. Done means avoiding the private tarfile APIs while preserving progress_filter's per-member redirection and skip behavior, with coverage for the security protections described here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- security, tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100