googleapis / googleapis/google-cloud-python

Retry should have a public way to access the predicate

Aperta
#15,026 2 commenti 0 reazioni 1 assegnatario Rivendicata da @tswast Vedi su GitHub
priority: p2 type: feature request
Lingua principale
Python
Stelle
5.4k
Fork
1.8k
Merge medio
3g 4h
PR unite (30g)
122

Descrizione

**Is your feature request related to a problem? Please describe.**

In BigQuery, there are some operations like "start a query" that can actually take multiple API requests. It's usually just `jobs.insert`, but if there's a conflict, then we also make calls to `jobs.get`. In addition, these have different predicates, such as wanting to retry `jobs.get` on 404 in this context but not others.

**Describe the solution you'd like**

Ideally, I could take a user-provided generic retry and tweak if for the given scenario. For example:

```python
get_job_retry = retry.with_predicate(
lambda exc: isinstance(exc, core_exceptions.NotFound)
# Reference the original retry to avoid recursion.
or retry._predicate(exc)
)
```

or

```python
def do_query_predicate(exc) -> bool:
if isinstance(exc, core_exceptions.RetryError):
exc = exc.cause

# Per https://github.com/googleapis/python-bigquery/issues/2134, sometimes
# we get a 404 error. In this case, if we get this far, assume that the job
# doesn't actually exist and try again. We can't add 404 to the default
# job_retry because that happens for errors like "this table does not
# exist", which probably won't resolve with a retry.
if isinstance(exc, core_exceptions.NotFound):
message = exc.message
# Don't try to retry table/dataset not found, just job not found.
# The URL contains jobs, so use whitespace to disambiguate.
return message is not None and " job" in message.lower()

# Reference the original job_retry to avoid recursion.
return job_retry._predicate(exc)
)
```

**Describe alternatives you've considered**

1. Just as I already have two retry objects in the parameters for the `query()` API, I could update it to have two more for these specific situations.
2. We could also add a method `with_additional_predicate` that adds an `or` clause + this function to the previous predicate.

**Additional context**

https://github.com/googleapis/python-bigquery/pull/2135 is an issue where I will likely add more "retry" parameters for finer-grained control, but would have preferred taking a user's retry object and amending the predicate.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.