Add back io::copy file to pipe optimization
Open
Nobody has claimed this yet.
C-feature-request
T-libs
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
So to sum up, I think we can add back optimization via:
- if the file is on squashfs/erofs, then it's immutable (detectable via
statfs - if the is sealed with
F_SEAL_WRITE(prevent modifying of existing contents, shrinking and appending is allowed) andF_SEAL_SEAL(not changing of seals), detectable viafcntl(fd, F_GET_SEALS) - if it's on read-only btrfs subvolume (e.g. snapshot), likewise we can do the same for zfs, nilfs, bcachefs
- If the file is
/dev/zero, or from other pseudo fs (e.g. procfs, sysfe), then I think we can assume it's immutable
Otherwise, we'd fallback to:
- read + write if the file is really small
- splice to an intermediate pipe with F_MOVE and then splice to the pipe without F_MOVE if the file is more than 1-2 pages
- if the fs supports reflink and file is really large, then we could create a O_TMPFILE on the system, copy_file_range to it and then splice it
Also, if the file is a named fifo, then just a splice without F_MOVE would work fine.
Originally posted by @NobodyXu in https://github.com/rust-lang/rust/issues/108283#issuecomment-2254439139
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the io::copy implementation and reading the referenced statfs, fcntl, splice, and copy_file_range APIs. Compare the proposed filesystem and file-type cases with the existing behavior; done means restoring the optimization while preserving the listed fallbacks and handling named FIFOs correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, rust
- Domain
- operating-systems, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100