`std::fs::copy` causes extra AppleDouble sidecar `._` files on exFAT due to `COPYFILE_ALL`
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
std::fs::copy on an exFAT drive creates ._ AppleDouble sidcar files if there are any extended attributes on files, as it uses COPYFILE_ALL. This problem was first reported in uv: https://github.com/astral-sh/uv/issues/18790.
This is different from python, which uses COPYFILE_DATA for shutil.copy instead:
https://github.com/python/cpython/blob/12828e5f98a47864e3f6d25fdd99c062fae28b1c/Lib/shutil.py#L319.
This is a problem extracting and then copying a directory, where there's suddenly extra files in the tree that shouldn't be there (https://github.com/astral-sh/uv/issues/18790). We would like to retain the mode (specifically, the executable bit), but avoid having extra files that cause problem in a walkdir-and-copy scheme. I couldn't find any discussion on COPYFILE_ALL in https://github.com/rust-lang/rust/pull/58901 (CC @ebarnard), hence I'm opening this issue.
This LLM-written script reproduces this on my macOS VM. It shows how std::fs::copy creates the sidecar file:
#!/usr/bin/env bash
# Reproducer: std::fs::copy creates ._ AppleDouble sidecar files on exFAT.
# Raw read/write and python avoid this.
WORKDIR=$(mktemp -d)
DMG="$WORKDIR/exfat.dmg"
VOL="$WORKDIR/mnt"
cleanup() { hdiutil detach "$VOL" 2>/dev/null; rm -rf "$WORKDIR"; }
trap cleanup EXIT
set -euo pipefail
mkdir -p "$VOL"
hdiutil create -size 64m -fs ExFAT -volname "EXFAT_UV" "$DMG" > /dev/null
hdiutil attach "$DMG" -mountpoint "$VOL" > /dev/null
SRC=$(mktemp)
echo "hello" > "$SRC"
xattr -w com.apple.quarantine "0081;deadbeef;test;0" "$SRC"
RDIR=$(mktemp -d)
cat > "$RDIR/main.rs" << 'EOF'
fn main() {
let args: Vec<String> = std::env::args().collect();
match args[1].as_str() {
"std" => { std::fs::copy(&args[2], &args[3]).unwrap(); }
"raw" => {
let meta = std::fs::metadata(&args[2]).unwrap();
let mut r = std::fs::File::open(&args[2]).unwrap();
let mut w = std::fs::File::create(&args[3]).unwrap();
std::io::copy(&mut r, &mut w).unwrap();
w.set_permissions(meta.permissions()).unwrap();
}
_ => panic!("usage: copier std|raw <src> <dst>"),
}
}
EOF
rustc "$RDIR/main.rs" -o "$RDIR/copier"
"$RDIR/copier" std "$SRC" "$VOL/test_std.txt"
echo "Rust std::fs::copy → sidecar ._ files: $(find "$VOL" -name '._test_std*' | wc -l | tr -d ' ')"
"$RDIR/copier" raw "$SRC" "$VOL/test_raw.txt"
echo "Rust raw read/write → sidecar ._ files: $(find "$VOL" -name '._test_raw*' | wc -l | tr -d ' ')"
python3 -c "import shutil,sys; shutil.copyfile(sys.argv[1], sys.argv[2])" "$SRC" "$VOL/test_py.txt"
echo "Python shutil → sidecar ._ files: $(find "$VOL" -name '._test_py*' | wc -l | tr -d ' ')"
rm -f "$SRC" && rm -rf "$RDIR"
Tested on 1.94.1, but the relevant code is in main (05cda351da24a2892d76fab6ed586aa348163d14).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in library/std/src/sys/fs/unix.rs around the COPYFILE_ALL call at line 2449 and compare the behavior with Python's COPYFILE_DATA approach. Run the supplied macOS exFAT reproducer to confirm the sidecar files and verify that the resulting copy retains the executable bit without creating AppleDouble files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100