GoogleCloudPlatform / GoogleCloudPlatform/gsutil

How to avoid files that are not found (WinError 3)?

Open
#1,163 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

(GSUTIL VS: 4.55) I was trying to back up my whole computer drive to google cloud storage, people mainly told me that was nonsensical, and while pondering it all, I decided just to do documents and desktop, and my goal was to use the google cloud SDK as I'm new to this and I like a challenge. So I type in:

`gsutil -m rsync -n -J -C -e -r -u -U C:\Users\~~~~\Documents gs://~~~~-pc/documents`

When I send that, I receive this message:

```
Caught non-retryable exception while listing file://C:\Users\~~~~~\Documents: [WinError 3] The system cannot find the path specified: 'C:\\Users\\~~~~~\\Documents\\~~~~~~~~~ Files\\2015 NEGOTIATIONIS\\2105 NEGOTIATIONS FILE - MAIN\\Implementation\\Temporary Transfers and Permanet Involuntary Force Rearragement\\Temporary Transfers and Permanent Involuntary Force Rearrangement FINAL October 3 2014.docx'
CommandException: Caught non-retryable exception - aborting rsync
```

Keep in mind ~ is me censoring my info.

Another Form told me to debug, and I copy and pasted the parts where an error occurred:

```
Caught non-retryable exception while listing file://C:\Users\~~~~\Documents: [WinError 3] The system cannot find the path specified: 'C:\\Users\\~~~~\\Documents\\~~~~ Files\\2015 NEGOTIATIONIS\\2105 NEGOTIATIONS FILE - MAIN\\Implementation\\Temporary Transfers and Permanet Involuntary Force Rearragement\\Temporary Transfers and Permanent Involuntary Force Rearrangement FINAL October 3 2014.docx'
Traceback (most recent call last):
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 653, in _ListUrlRootFunc
out_file)
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 903, in _BatchSort
current_chunk = sorted(islice(in_iter, buffer_size))
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 734, in _FieldedListingIterator
for blr in iterator:
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\wildcard_iterator.py", line 701, in IterObjects
bucket_listing_fields=bucket_listing_fields):
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\wildcard_iterator.py", line 721, in IterAll
bucket_listing_fields=bucket_listing_fields):
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\wildcard_iterator.py", line 611, in __iter__
blr_object = _GetFileObject(filepath) if include_size else None
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\wildcard_iterator.py", line 531, in _GetFileObject
return apitools_messages.Object(size=os.path.getsize(filepath))
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\bundledpython\lib\genericpath.py", line 50, in getsize
return os.stat(filename).st_size
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\~~~~\\Documents\\~~~~ Files\\2015 NEGOTIATIONIS\\2105 NEGOTIATIONS FILE - MAIN\\Implementation\\Temporary Transfers and Permanet Involuntary Force Rearragement\\Temporary Transfers and Permanent Involuntary Force Rearrangement FINAL October 3 2014.docx'

DEBUG: Exception stack trace:
Traceback (most recent call last):
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\__main__.py", line 634, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\command_runner.py", line 411, in RunNamedCommand
return_code = command_inst.RunCommand()
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 1657, in RunCommand
diff_iterator = _DiffIterator(self, src_url, dst_url)
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 999, in __init__
raise CommandException('Caught non-retryable exception - aborting rsync')
gslib.exception.CommandException: CommandException: Caught non-retryable exception - aborting rsync
```

I am very new to all this, so please help me figure this out!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Windows command and read gslib/commands/rsync.py around _ListUrlRootFunc and _DiffIterator. Then inspect gslib/wildcard_iterator.py around _GetFileObject and the failing os.path.getsize call. Done means establishing the expected behavior when a local file disappears or cannot be statted during rsync, with a focused regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.