googleapis / googleapis/google-cloud-python

Retry should have a public way to access the predicate

Aberta
#15,026 2 comentários 0 reações 1 responsável Ver no GitHub

@tswast já está trabalhando nisso.

Desde 27/5/2026.

priority: p2 type: feature request
Linguagem predominante
Python
Estrelas
5.4k
Forks
1.8k
Merge médio
2d 23h
PRs com merge (30d)
123

Descrição

Is your feature request related to a problem? Please describe.

In BigQuery, there are some operations like "start a query" that can actually take multiple API requests. It's usually just jobs.insert, but if there's a conflict, then we also make calls to jobs.get. In addition, these have different predicates, such as wanting to retry jobs.get on 404 in this context but not others.

Describe the solution you'd like

Ideally, I could take a user-provided generic retry and tweak if for the given scenario. For example:

get_job_retry = retry.with_predicate(
                        lambda exc: isinstance(exc, core_exceptions.NotFound)
                        # Reference the original retry to avoid recursion.
                        or retry._predicate(exc)
)

or

def do_query_predicate(exc) -> bool:
            if isinstance(exc, core_exceptions.RetryError):
                exc = exc.cause

            # Per https://github.com/googleapis/python-bigquery/issues/2134, sometimes
            # we get a 404 error. In this case, if we get this far, assume that the job
            # doesn't actually exist and try again. We can't add 404 to the default
            # job_retry because that happens for errors like "this table does not
            # exist", which probably won't resolve with a retry.
            if isinstance(exc, core_exceptions.NotFound):
                message = exc.message
                # Don't try to retry table/dataset not found, just job not found.
                # The URL contains jobs, so use whitespace to disambiguate.
                return message is not None and " job" in message.lower()

            # Reference the original job_retry to avoid recursion.
            return job_retry._predicate(exc)
)

Describe alternatives you've considered

  1. Just as I already have two retry objects in the parameters for the query() API, I could update it to have two more for these specific situations.
  2. We could also add a method with_additional_predicate that adds an or clause + this function to the previous predicate.

Additional context

https://github.com/googleapis/python-bigquery/pull/2135 is an issue where I will likely add more "retry" parameters for finer-grained control, but would have preferred taking a user's retry object and amending the predicate.

Guia de contribuição

Abrir o guia de contribuição

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.