googleapis / googleapis/google-cloud-python

Retry should have a public way to access the predicate

オープン
#15,026 コメント 2 件 リアクション 0 件 担当者 1 名 @tswast が担当を希望しています GitHub で見る
priority: p2 type: feature request
主要言語
Python
スター
5.4k
フォーク
1.8k
平均マージ
3日 4時間
マージ済み PR(30日)
122

説明

**Is your feature request related to a problem? Please describe.**

In BigQuery, there are some operations like "start a query" that can actually take multiple API requests. It's usually just `jobs.insert`, but if there's a conflict, then we also make calls to `jobs.get`. In addition, these have different predicates, such as wanting to retry `jobs.get` on 404 in this context but not others.

**Describe the solution you'd like**

Ideally, I could take a user-provided generic retry and tweak if for the given scenario. For example:

```python
get_job_retry = retry.with_predicate(
lambda exc: isinstance(exc, core_exceptions.NotFound)
# Reference the original retry to avoid recursion.
or retry._predicate(exc)
)
```

or

```python
def do_query_predicate(exc) -> bool:
if isinstance(exc, core_exceptions.RetryError):
exc = exc.cause

# Per https://github.com/googleapis/python-bigquery/issues/2134, sometimes
# we get a 404 error. In this case, if we get this far, assume that the job
# doesn't actually exist and try again. We can't add 404 to the default
# job_retry because that happens for errors like "this table does not
# exist", which probably won't resolve with a retry.
if isinstance(exc, core_exceptions.NotFound):
message = exc.message
# Don't try to retry table/dataset not found, just job not found.
# The URL contains jobs, so use whitespace to disambiguate.
return message is not None and " job" in message.lower()

# Reference the original job_retry to avoid recursion.
return job_retry._predicate(exc)
)
```

**Describe alternatives you've considered**

1. Just as I already have two retry objects in the parameters for the `query()` API, I could update it to have two more for these specific situations.
2. We could also add a method `with_additional_predicate` that adds an `or` clause + this function to the previous predicate.

**Additional context**

https://github.com/googleapis/python-bigquery/pull/2135 is an issue where I will likely add more "retry" parameters for finer-grained control, but would have preferred taking a user's retry object and amending the predicate.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。