googleapis / googleapis/google-cloud-python
Retry should have a public way to access the predicate
- Lenguaje dominante
- Python
- Estrellas
- 5.4k
- Forks
- 1.8k
- Merge medio
- 3 d 4 h
- PR fusionados (30 d)
- 122
Descripción
**Is your feature request related to a problem? Please describe.**
In BigQuery, there are some operations like "start a query" that can actually take multiple API requests. It's usually just `jobs.insert`, but if there's a conflict, then we also make calls to `jobs.get`. In addition, these have different predicates, such as wanting to retry `jobs.get` on 404 in this context but not others.
**Describe the solution you'd like**
Ideally, I could take a user-provided generic retry and tweak if for the given scenario. For example:
```python
get_job_retry = retry.with_predicate(
lambda exc: isinstance(exc, core_exceptions.NotFound)
# Reference the original retry to avoid recursion.
or retry._predicate(exc)
)
```
or
```python
def do_query_predicate(exc) -> bool:
if isinstance(exc, core_exceptions.RetryError):
exc = exc.cause
# Per https://github.com/googleapis/python-bigquery/issues/2134, sometimes
# we get a 404 error. In this case, if we get this far, assume that the job
# doesn't actually exist and try again. We can't add 404 to the default
# job_retry because that happens for errors like "this table does not
# exist", which probably won't resolve with a retry.
if isinstance(exc, core_exceptions.NotFound):
message = exc.message
# Don't try to retry table/dataset not found, just job not found.
# The URL contains jobs, so use whitespace to disambiguate.
return message is not None and " job" in message.lower()
# Reference the original job_retry to avoid recursion.
return job_retry._predicate(exc)
)
```
**Describe alternatives you've considered**
1. Just as I already have two retry objects in the parameters for the `query()` API, I could update it to have two more for these specific situations.
2. We could also add a method `with_additional_predicate` that adds an `or` clause + this function to the previous predicate.
**Additional context**
https://github.com/googleapis/python-bigquery/pull/2135 is an issue where I will likely add more "retry" parameters for finer-grained control, but would have preferred taking a user's retry object and amending the predicate.
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.