python / python/cpython

tarfile indeterminate TarInfo.size when PAX headers contain `size` and `GNU.sparse.realsize` keys at the same time

未關閉
#136,601 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

stdlib type-bug
主要語言
Python
星號
77.2k
分支
35.9k
PR 合併指標
PR 指標待擷取

描述

Bug report

Bug description:

Hello,

I am currently debugging this issue.

I have noticed that the bug can be reproduced when the problematic file is truncated to 9 GiB B but it does not happen when truncated to 8 GiB.

The problem seems to be that the next member offset is computed wrong. It seems to point 512 B after the correct TAR header, which, in this case, points into the data for the extended attributes such as 30 mtime=1752348[...].

One of the differences seems to be this code part, which is not hit for the working case:

https://github.com/python/cpython/blob/47b01da4ccedd9c00fad4325b3e87d7732abeb6d/Lib/tarfile.py#L1562-L1569

While looking into the line above, i.e., into _apply_pax_info, I noticed that there is no definite order for applying the size even though it can appear multiple times!

https://github.com/python/cpython/blob/47b01da4ccedd9c00fad4325b3e87d7732abeb6d/Lib/tarfile.py#L1615-L1634

In the non-working case, the PAX headers look like this:

{'GNU.sparse.major': '1',
 'GNU.sparse.minor': '0',
 'GNU.sparse.name': 'userdata',
 'GNU.sparse.realsize': '9663676416',
 'atime': '1752349406.975921575',
 'ctime': '1752349534.57652562',
 'mtime': '1752349534.57652562',
 'size': '9602318848'}

I.e, the size member first gets set to GNU.sparse.realsize and then to size. The debug output looks like this:

[_apply_pax_info] SET SIZE to: 9663676416 from key: GNU.sparse.realsize
[_apply_pax_info] SET SIZE to: 9602318848 from key: size
[_apply_pax_info] SET key to: 1752349534.5765257 from key: mtime

Is it specified that the order of the PAX headers must always be this way? Else, one might just as well encounter it like this:

{'atime': '1752349406.975921575',
 'ctime': '1752349534.57652562',
 'mtime': '1752349534.57652562',
 'size': '9602318848',
 'GNU.sparse.major': '1',
 'GNU.sparse.minor': '0',
 'GNU.sparse.name': 'userdata',
 'GNU.sparse.realsize': '9663676416'}

and either one of these orders would be a bug.

The working case does not have this ambiguity:

{'GNU.sparse.major': '1',
 'GNU.sparse.minor': '0',
 'GNU.sparse.name': 'userdata',
 'GNU.sparse.realsize': '8589934592',
 'atime': '1752349538.445543898',
 'ctime': '1752351104.53673501',
 'mtime': '1752351104.53673501'}

the debug output looks like this:

[_apply_pax_info] SET SIZE to: 8589934592 from key: GNU.sparse.realsize
[_apply_pax_info] SET key to: 1752351104.536735 from key: mtime

I.e., even if the is no ordering problem, there already are different semantics for the TarInfo.size member as one will contain GNU.sparse.realsize and the other will contain [PAXHeader.]size.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs
  • gh-136622

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

從 Lib/tarfile.py 開始,尤其關注 _apply_pax_info 以及所引用行附近的大小處理。重現 8 GiB 和 9 GiB 的不同情況,並檢查關聯的 PR gh-136622。完成的標準是:PAX 大小語義和 TarInfo.size 的行為都明確無歧義,並由適當的回歸測試涵蓋。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
tooling
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
20/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。