googleapis / googleapis/google-cloud-python

Retry should have a public way to access the predicate

Đang mở
#15,026 2 bình luận 0 reaction 1 người được giao Được @tswast nhận Xem trên GitHub
priority: p2 type: feature request
Ngôn ngữ chính
Python
Star
5.4k
Fork
1.8k
Merge trung bình
3 ngày 4 giờ
Pull request đã merge (30 ngày)
122

Mô tả

**Is your feature request related to a problem? Please describe.**

In BigQuery, there are some operations like "start a query" that can actually take multiple API requests. It's usually just `jobs.insert`, but if there's a conflict, then we also make calls to `jobs.get`. In addition, these have different predicates, such as wanting to retry `jobs.get` on 404 in this context but not others.

**Describe the solution you'd like**

Ideally, I could take a user-provided generic retry and tweak if for the given scenario. For example:

```python
get_job_retry = retry.with_predicate(
lambda exc: isinstance(exc, core_exceptions.NotFound)
# Reference the original retry to avoid recursion.
or retry._predicate(exc)
)
```

or

```python
def do_query_predicate(exc) -> bool:
if isinstance(exc, core_exceptions.RetryError):
exc = exc.cause

# Per https://github.com/googleapis/python-bigquery/issues/2134, sometimes
# we get a 404 error. In this case, if we get this far, assume that the job
# doesn't actually exist and try again. We can't add 404 to the default
# job_retry because that happens for errors like "this table does not
# exist", which probably won't resolve with a retry.
if isinstance(exc, core_exceptions.NotFound):
message = exc.message
# Don't try to retry table/dataset not found, just job not found.
# The URL contains jobs, so use whitespace to disambiguate.
return message is not None and " job" in message.lower()

# Reference the original job_retry to avoid recursion.
return job_retry._predicate(exc)
)
```

**Describe alternatives you've considered**

1. Just as I already have two retry objects in the parameters for the `query()` API, I could update it to have two more for these specific situations.
2. We could also add a method `with_additional_predicate` that adds an `or` clause + this function to the previous predicate.

**Additional context**

https://github.com/googleapis/python-bigquery/pull/2135 is an issue where I will likely add more "retry" parameters for finer-grained control, but would have preferred taking a user's retry object and amending the predicate.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.