fsspec / fsspec/filesystem_spec
copy directory contains symbolic to subdir to directory twice bigger
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
src dir:
data/2025-05-16
├── sparse_embeddings -> model/sparse_embeddings
└── model/sparse_embeddings/
when use local fs to copy, the sparse_embeddings symbolic link will copy model/sparse_embeddings data duplicated. As a result, the data/2025-05-16 directory will be twice bigger than origin.
from fsspec.implementations.local import LocalFileSystem
fs = LocalFileSystem()
fs.copy("data/2025-05-16/", "models/2025-05-16/", recursive=True)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the LocalFileSystem.copy entry point and reproduce the recursive copy with the shown directory layout, including the symbolic link to model/sparse_embeddings. Trace how recursive copying handles symbolic links; done means the destination does not duplicate the symlink target data and the reproduction no longer makes the copied directory twice as large.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100