GoogleCloudPlatform / GoogleCloudPlatform/gsutil
rsync Copying mtime from src to dst for gs://... gives BadRequestException: 400 InvalidArgument
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
For years, we've been syncing a local folder to a gcs bucket using following gsutil command:
```gsutil -m rsync -r /my/local/folder gs://my_bucket```
Today we upgraded from gsutil 4.19 to 4.42.
Well aware that the change detection algorithm uses the mtime as from version 4.20.
When running rsync now, gsutil seems to want to set the mtime on the file in the gcs bucket. When checking, indeed, the mtime in the gcs bucket differs from the local mtime: the gcs mtime seems to be the time of creation of the file in gcs (using rsync!), which is a bit later than the original mtime of the local filesystem.
Now, gsutil seems to be wanting to set the local mtime on each gcs file. It shouldn't do this, as the gcs file is "more recent" than the local file!
However, for each of the 30k files, gsutil shows an error message. The mtime is indeed not changed:
```
Copying mtime from src to dst for gs://my_bucket/data/0/9953
BadRequestException: 400 InvalidArgument
InvalidArgumentInvalid argument.
```
Using Python 3.6.7 or Python 2.7.15 doesn't change it.
More info:
Stat of a local file:
```
File: 9999
Size: 8360612 Blocks: 16336 IO Block: 4096 regular file
Device: 902h/2306d Inode: 43532596 Links: 1
Access: (0770/-rwxrwx---) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2019-08-10 12:57:09.576056289 +0200
Modify: 2016-11-20 03:14:39.133679318 +0100
Change: 2019-06-18 12:12:00.293338488 +0200
Birth: -
```
gsutil stat of the same file in gcs:
```
Creation time: Sun, 20 Nov 2016 03:21:02 GMT
Storage class: NEARLINE
Cache-Control: private, max-age=0
Content-Encoding: identity
Content-Language: en
Content-Length: 6098480
Content-Type: application/octet-stream
Hash (crc32c): fNwrBw==
Hash (md5): C+L4duQPRWmEftGnkZSfgQ==
ETag: 0be2f876e40f4569847ed1a791949f81
Generation: 1479612062879460
Metageneration: 1
```
Running gsutil with the `-u` flag doesn't seem to change the behavior.
Running gsutil with the `-n` flag gives messages like:
```Would set mtime for gs://my_bucket/data/0/9953```
Contributor guide
Assessment
This issue has not been assessed yet.