fish-shell / fish-shell/fish-shell
`**` glob witnessing dataless directories materializes them on macOS
- Dominant language
- Rust
- Stars
- 34.2k
- Forks
- 2.4k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 13
Description
Files that exist "dataless" on disk are being forced to download and sync locally just by iterating over them with `**`.
This affects stuff like iCloud paths, likely dropbox, Google Drive, box accounts etc. (assuming they now use this) and definitely S3/DAV/FTP/whatever shares through [Strongsync](https://www.expandrive.com/strongsync/) which uses the [FileProvider API](https://developer.apple.com/documentation/fileprovider).
"dataless" is a state supported by APFS in which a file or directory is a placeholder, and its children (for a directory) or content (for a file) will be fetched when the user tries to read it.
`ls -l%` is an easy way to see if files are dataless or not.
```
-% Distinguish dataless files and directories with a '%' character in long (-l) output, and
don't materialize dataless directories when listing them.
```
I first noticed this when trying to determine why `**` in my home directory could take minutes. After attempting to debug it for a while I noticed all of a sudden it was quite fast. ~~But my hard disk was almost empty.~~
```fish
~/L/C/Strongsync-SPMDrive►echo *
bugreceipt.png Inspection Entry.webloc Inspection Record.webloc log_0D1CD0_1-21.webloc OrgData-smallpartsmfg.com-1p2uljz3ras55u-20160720.csv.webloc Shared Drives Shared with Me sonicwall-TZ_400-6_2_3_1-19n.exp SPM techSupport_0D1CD0_1-21.wri test.pdf textual.mov Untitled spreadsheet.webloc
~/L/C/Strongsync-SPMDrive►ls -l%
total 0
-rw-------% 1 floam staff 317 May 6 2015 Inspection Entry.webloc
-rw-------% 1 floam staff 324 Oct 29 2014 Inspection Record.webloc
-rw-------% 1 floam staff 324 Oct 6 2016 OrgData-smallpartsmfg.com-1p2uljz3ras55u-20160720.csv.webloc
drwx------% 20 floam staff 640 Sep 29 2015 SPM/
drwx------% 2 floam staff 64 Nov 2 00:13 Shared Drives/
drwx------% 10 floam staff 320 Nov 2 00:13 Shared with Me/
-rw-------% 1 floam staff 324 Oct 29 2014 Untitled spreadsheet.webloc
-rw-------% 1 floam staff 78132 Apr 23 2016 bugreceipt.png
-rw-------% 1 floam staff 324 Jan 21 2016 log_0D1CD0_1-21.webloc
-rw-------% 1 floam staff 808062 Jan 21 2016 sonicwall-TZ_400-6_2_3_1-19n.exp
-rw-------% 1 floam staff 738966 Jan 21 2016 techSupport_0D1CD0_1-21.wri
-rw-------% 1 floam staff 492607 Apr 1 2016 test.pdf
-rw-------% 1 floam staff 8467525 Apr 18 2016 textual.mov
~/L/C/Strongsync-SPMDrive►echo **
bugreceipt.png Inspection Entry.webloc Inspection Record.webloc log_0D1CD0_1-21.webloc OrgData-smallpartsmfg.com-1p2uljz3ras55u-20160720.csv.webloc Shared Drives Shared with Me Shared with Me/BOTTOM.step Shared with Me/image0.jpeg Shared with Me/image1.jpeg Shared with Me/image2.jpeg Shared with Me/image3.jpeg Shared with Me/image5.png Shared with Me/sw_tz-400_eng_6.2.5.0_6.2.5_15n_858612.sig Shared with Me/TOP.step sonicwall-TZ_400-6_2_3_1-19n.exp SPM SPM/IMG_0393.JPG SPM/IMG_0394.JPG SPM/IMG_0395.JPG SPM/IMG_0396.JPG SPM/IMG_0397.JPG SPM/IMG_0398.JPG SPM/IMG_0399.JPG SPM/IMG_0400.JPG SPM/IMG_0401.JPG SPM/IMG_0402.JPG SPM/IMG_0403.JPG SPM/IMG_0404.JPG SPM/IMG_0405.JPG SPM/IMG_0406.JPG SPM/IMG_0407.JPG SPM/IMG_0408.JPG SPM/IMG_0412.JPG SPM/IMG_0413.JPG techSupport_0D1CD0_1-21.wri test.pdf textual.mov Untitled spreadsheet.webloc
~/L/C/Strongsync-SPMDrive►echo $CMD_DURATION
9486
ls -l%
total 0
-rw-------% 1 floam staff 317 May 6 2015 Inspection Entry.webloc
-rw-------% 1 floam staff 324 Oct 29 2014 Inspection Record.webloc
-rw-------% 1 floam staff 324 Oct 6 2016 OrgData-smallpartsmfg.com-1p2uljz3ras55u-20160720.csv.webloc
drwx------@ 20 floam staff 640 Sep 29 2015 SPM/
drwx------@ 2 floam staff 64 Nov 2 00:13 Shared Drives/
drwx------@ 10 floam staff 320 Nov 2 00:13 Shared with Me/
-rw-------% 1 floam staff 324 Oct 29 2014 Untitled spreadsheet.webloc
-rw-------% 1 floam staff 78132 Apr 23 2016 bugreceipt.png
-rw-------% 1 floam staff 324 Jan 21 2016 log_0D1CD0_1-21.webloc
-rw-------% 1 floam staff 808062 Jan 21 2016 sonicwall-TZ_400-6_2_3_1-19n.exp
-rw-------% 1 floam staff 738966 Jan 21 2016 techSupport_0D1CD0_1-21.wri
-rw-------% 1 floam staff 492607 Apr 1 2016 test.pdf
-rw-------% 1 floam staff 8467525 Apr 18 2016 textual.mov
```
Notice the three directories that have apparently forced remote file servers to be queried, etc.
Neither `ls` (except -l sans -%, see quoted manual above), `du`, nor `echo **` on zsh has this behavior. We should figure out how to avoid it.
Contributor guide
Assessment
This issue has not been assessed yet.