bloomreach / bloomreach/s4cmd

get of a file with "+" in the name fails with [Runtime Failure] for google.cloud

Open
#212 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.4k
Forks
208
PR merge metrics
No merged PRs in 30d

Description

$ s4cmd -c 32 --debug get --endpoint-url https://storage.googleapis.com s3://mib_dump/test/file_+.txt /tmp/abcd
(D)s4cmd:631 read S3 keys from environment
(D)s4cmd:129 >> get_handler(<main.CommandHandler object at 0x7f8c17a52c10>, ['get', 's3://mib_dump/test/file
+.txt', '/tmp/abcd'])
(D)s4cmd:129 >> validate(<main.CommandHandler object at 0x7f8c17a52c10>, 'cmd|s3|local', ['get', 's3://mib_dump/test/file
+.txt', '/tmp/abcd'])
(D)s4cmd:131 << validate(<main.CommandHandler object at 0x7f8c17a52c10>, 'cmd|s3|local', ['get', 's3://mib_dump/test/file
+.txt', '/tmp/abcd']): None
(D)s4cmd:129 >> get_files(<main.S3Handler object at 0x7f8c17a67150>, 's3://mib_dump/test/file
+.txt', '/tmp/abcd')
(D)s4cmd:129 >> s3walk(<main.S3Handler object at 0x7f8c17a67150>, 's3://mib_dump/test/file
+.txt', True)
(D)s4cmd:129 >> s3walk(<ThreadUtil(Thread-33, started daemon 140237611177728)>, <main.S3URL instance at 0x7f8bdc45b758>, 'test/file
+.txt', 'test/file+.txt', [])
(D)s4cmd:402 >> S3APICALL get_paginator('list_objects')
(D)s4cmd:404 << S3APICALL get_paginator('list_objects'): <botocore.client.S3.Paginator.ListObjects object at 0x7f8b7c647590>
(D)s4cmd:129 >> partial_match(<ThreadUtil(Thread-33, started daemon 140237611177728)>, 'test/file
%2B_.txt', 'test/file_+.txt')
(D)s4cmd:131 << partial_match(<ThreadUtil(Thread-33, started daemon 140237611177728)>, 'test/file
%2B_.txt', 'test/file_+.txt'): False
(D)s4cmd:131 << s3walk(<ThreadUtil(Thread-33, started daemon 140237611177728)>, <main.S3URL instance at 0x7f8bdc45b758>, 'test/file
+.txt', 'test/file+.txt', []): None
(D)s4cmd:131 << s3walk(<main.S3Handler object at 0x7f8c17a67150>, 's3://mib_dump/test/file
+_.txt', True): []
(E)s4cmd:183 [Runtime Failure] Source doesn't exist.

$ s4cmd --version
s4cmd version 2.1.0
$ python2 --version
Python 2.7.16

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the command shown with s4cmd 2.1.0 against the Google Cloud endpoint, then inspect the traced s3walk and partial_match calls. The issue is done when getting an object named file_+_.txt succeeds instead of reporting that the source does not exist.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.