GoogleCloudPlatform / GoogleCloudPlatform/gsutil

feature request: warning about coldline retrieval costs

Open
#552 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

I hope this is the correct place to file this feature/enhancement request.

I would like to suggest that gsutil generate a warning if a bucket is of type "NEARLINE" but the object in that bucket is of type "COLDLINE". As operating on a bucket that you believe contains nearline objects but instead contains coldline objects can result in large unexpected bills for a naive operator such as myself. I managed to spend $6k+ with one 'gsutil rsync' command.

For example:

$ gsutil ls -L -b gs://name-redacted
gs://name-redacted/ :
**Storage class: NEARLINE**
Location constraint: US
Versioning enabled: None
Logging configuration: None
Website configuration: None
CORS configuration: None
**Lifecycle configuration: Present**
Requester Pays enabled: None
Labels: None
Default KMS key: None
Time created: Thu, 26 May 2016 22:51:58 GMT
Time updated: Mon, 04 Jun 2018 21:31:03 GMT
Metageneration: 14
...

$ gsutil lifecycle get gs://name-redacted
{"rule": [{"action": {"storageClass": "COLDLINE", "type": "SetStorageClass"}, "condition": {"matchesStorageClass": ["NEARLINE"]}}]}

Examining objects in the bucket shows they are coldline:

$ gsutil ls -L gs://name-redacted/logs/2016-05-29-13-34-45-etc
gs://name-redacted/logs/2016-05-29-13-34-45-etc:
Creation time: Tue, 31 May 2016 19:24:42 GMT
Update time: Sat, 28 Oct 2017 09:07:40 GMT
Storage class update time:Sat, 28 Oct 2017 09:07:40 GMT
Storage class: COLDLINE
...

I now double-checked and I do see that if you drill down in the bucket to look at the objects, the do show class "nearline".

Just hoping this doesn't bite other people.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.