Does recursive get work?
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 209
- PR merge metrics
- No merged PRs in 30d
Description
I'm simply trying to download a bunch of files from s3, in a recursive manner. I am using the common convention that the s3 objects and prefixes can be viewed as directories and files, using "/" as the separator.
the following works using s3cmd:
s3cmd get -rf s3://bucket_name/ ./dirname/
the following does not work with s4cmd, using the exact same source/target locations:
s4cmd get -rf s3://bucket_name/ ./dirname/
I get errors like:
[Runtime Failure] Download Failure: , Source: s3://bucket/subdir/.
[Runtime Failure] Download Failure: , Source: s3://bucket/subdir/subdir2/subdir3a/.
[Runtime Failure] Download Failure: , Source: s3://bucket/subdir/subdir2/.
[Runtime Failure] Download Failure: , Source: s3://bucket/subdir/subdir2/subdir3b/.
[OSError] 2: No such file or directory
[OSError] 2: No such file or directory
[OSError] 2: No such file or directory
....
Is not s4cmd supposed to be drop-in replacement for s3cmd for these types of things? Does anyone have any troubleshooting advice? Can you point me to a public-access bucket on s3 where I can reproduce a recursive download using s4cmd?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the reported `s4cmd get -rf` command against a bucket containing nested prefixes, comparing it with the working `s3cmd` command. Trace the recursive download entry point and the reported missing-directory failures; done means nested objects download successfully without the shown errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cli, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100