GoogleCloudPlatform / GoogleCloudPlatform/gsutil
How to avoid files that are not found (WinError 3)?
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
(GSUTIL VS: 4.55) I was trying to back up my whole computer drive to google cloud storage, people mainly told me that was nonsensical, and while pondering it all, I decided just to do documents and desktop, and my goal was to use the google cloud SDK as I'm new to this and I like a challenge. So I type in:
`gsutil -m rsync -n -J -C -e -r -u -U C:\Users\~~~~\Documents gs://~~~~-pc/documents`
When I send that, I receive this message:
```
Caught non-retryable exception while listing file://C:\Users\~~~~~\Documents: [WinError 3] The system cannot find the path specified: 'C:\\Users\\~~~~~\\Documents\\~~~~~~~~~ Files\\2015 NEGOTIATIONIS\\2105 NEGOTIATIONS FILE - MAIN\\Implementation\\Temporary Transfers and Permanet Involuntary Force Rearragement\\Temporary Transfers and Permanent Involuntary Force Rearrangement FINAL October 3 2014.docx'
CommandException: Caught non-retryable exception - aborting rsync
```
Keep in mind ~ is me censoring my info.
Another Form told me to debug, and I copy and pasted the parts where an error occurred:
```
Caught non-retryable exception while listing file://C:\Users\~~~~\Documents: [WinError 3] The system cannot find the path specified: 'C:\\Users\\~~~~\\Documents\\~~~~ Files\\2015 NEGOTIATIONIS\\2105 NEGOTIATIONS FILE - MAIN\\Implementation\\Temporary Transfers and Permanet Involuntary Force Rearragement\\Temporary Transfers and Permanent Involuntary Force Rearrangement FINAL October 3 2014.docx'
Traceback (most recent call last):
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 653, in _ListUrlRootFunc
out_file)
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 903, in _BatchSort
current_chunk = sorted(islice(in_iter, buffer_size))
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 734, in _FieldedListingIterator
for blr in iterator:
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\wildcard_iterator.py", line 701, in IterObjects
bucket_listing_fields=bucket_listing_fields):
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\wildcard_iterator.py", line 721, in IterAll
bucket_listing_fields=bucket_listing_fields):
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\wildcard_iterator.py", line 611, in __iter__
blr_object = _GetFileObject(filepath) if include_size else None
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\wildcard_iterator.py", line 531, in _GetFileObject
return apitools_messages.Object(size=os.path.getsize(filepath))
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\bundledpython\lib\genericpath.py", line 50, in getsize
return os.stat(filename).st_size
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\~~~~\\Documents\\~~~~ Files\\2015 NEGOTIATIONIS\\2105 NEGOTIATIONS FILE - MAIN\\Implementation\\Temporary Transfers and Permanet Involuntary Force Rearragement\\Temporary Transfers and Permanent Involuntary Force Rearrangement FINAL October 3 2014.docx'
DEBUG: Exception stack trace:
Traceback (most recent call last):
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\__main__.py", line 634, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\command_runner.py", line 411, in RunNamedCommand
return_code = command_inst.RunCommand()
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 1657, in RunCommand
diff_iterator = _DiffIterator(self, src_url, dst_url)
File "C:\Users\~~~~\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\commands\rsync.py", line 999, in __init__
raise CommandException('Caught non-retryable exception - aborting rsync')
gslib.exception.CommandException: CommandException: Caught non-retryable exception - aborting rsync
```
I am very new to all this, so please help me figure this out!
Contributor guide
Research direction
Start by reproducing the Windows command and read gslib/commands/rsync.py around _ListUrlRootFunc and _DiffIterator. Then inspect gslib/wildcard_iterator.py around _GetFileObject and the failing os.path.getsize call. Done means establishing the expected behavior when a local file disappears or cannot be statted during rsync, with a focused regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100