Infinite recursion in walk() when S3 bucket contains object with key /
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
We've run into a reproducible issue where calling walk() on an S3 bucket causes infinite recursion if the bucket contains an object with the key /
This happens because:
- S3 allows an object to be created with the key /
- ls() returns it as a "directory" (i.e. common prefix /)
- walk() sees that as a subdirectory and recurses into it
- But that "subdirectory" is just the bucket root again → leading to infinite recursion
- This eventually leads to a RecursionError: maximum recursion depth exceeded.
Let us know if this is something you'd consider filtering out at the s3fs level. Happy to provide a test case or additional details!
Thanks for the great library 🙏
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Python implementation of walk() and its use of ls(), then reproduce the case with an S3 object whose key is /. Trace how the returned common prefix is treated as a child directory; done means the case no longer recurses indefinitely and a regression test covers it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100