shutil.copy2 does not copy sparse files efficiently
Open
Nobody has claimed this yet.
type-feature
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
$ dd if=/dev/zero of=sparse_original bs=1 count=1 oseek=1M
1+0 records in
1+0 records out
1 byte copied, 2.7367e-05 s, 36.5 kB/s
$ cp sparse_original sparse_cp
$ python3 -c 'import shutil; shutil.copy2("sparse_original", "sparse_pycopy2");'
$ du sparse_original sparse_cp sparse_pycopy2
4 sparse_original
4 sparse_cp
1028 sparse_pycopy2
Ideally shutil.copy2() should be capable of copying sparse files without unnecessarily expanding them (this is the default behavior of cp).
This could potentially be implemented using lseek(2) SEEK_DATA and SEEK_HOLE.
CPython versions tested on:
3.12
Operating systems tested on:
Linux
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the shutil.copy2 entry point and reproduce the sparse-file example on Linux. Investigate lseek(2) with SEEK_DATA and SEEK_HOLE, then verify that the copied file preserves holes like cp instead of expanding them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, python
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100