GoogleCloudPlatform / GoogleCloudPlatform/gsutil

Add ability to print details for each error when running in `-m` mode without debug mode (`-D`)

Aperta
#653 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Feature Request
Lingua principale
Python
Stelle
918
Fork
335
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Internal context for Google devs: I'm filing this to mirror the discussion in https://issuetracker.google.com/issues/126268394.

There are a variety of use cases in which this functionality would be helpful. As an example: When performing an object deletion, gsutil may receive a 503 response or hit a `socket.error` exception if the request hits the GCS backend, succeeds, but then *something bad* happens before the 200 response can reach the client. In this case, gsutil retries the request, then gets back a 404 for the retry because the object has already been deleted. From gsutil's current perspective, we should report all permanent failures to the user and treat them as errors. However, when looking at this and considering idempotency, we might want to treat 404s as OK, as they don't result in a change in the state of any buckets, objects, etc. In absence of some notion of idempotent requests at the GCS service level, the next best thing would be to either allow an option for gsutil to treat 404s as OK for certain operations (e.g. resource deletions), or to print out exception details for permanent failures so that we can manually examine them (and maybe determine that they were acceptable, as in the case of the above-mentioned 404). This feature request is for the latter.

Currently, when we haven't told gsutil to continue upon hitting an error (e.g. using the top-level `-m` option, `cp`'s `-c` option, etc.), it will print out a stack trace for the first permanent failure it encounters, then exit... but when running in parallel mode (using `-m`), the exceptions are suppressed and summarized at the end (e.g. "X files/objects could not be removed") - this FR is asking for an option that could be supplied along with `-m` that would make gsutil print out details about all the permanent failures it encounters (probably as they're encountered, rather than at the end). Note that while this is already achievable by using the top-level `-D` option, that also dumps a LOT of extra log entries to the console, most of which are probably uninteresting in the majority of cases.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.