Expired authentication credentials are reported as HTTP 503 (Service Unavailable) errors

Open
#18,419 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
65/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
google-cloud, python

Research direction

Reproduce the failure from stormware/google/secrets.py at line 71 using expired credentials, then trace the generated Secret Manager client through google/api_core/gapic_v1/method.py and google/api_core/retry/retry_unary.py. Done means expired-credential failures surface immediately as an authentication error rather than being retried as HTTP 503 until the 60-second timeout.

Written by the indexing model from the issue text.

Description

triage me type: bug
Determine this is the right repository
  • I determined this is the correct repository in which to report this bug.
Summary of the issue

Context
Whenever we try to use the SDK clients with expired authentication credentials our programs halt for 60 seconds. This is because such requests get a 503 HTTP response back (which is not the correct response), so the library keeps retrying until the default timeout (which is 60 seconds).

Expected Behavior:
We expect SDK clients to fail immediately when credentials are expired.

Actual Behavior:
SDK clients do not fail immediately when credentials are expired.

API client name and version

google-cloud-secret-manager==2.30.0

Reproduction steps: code

See https://github.com/logikal-io/stormware/blob/main/stormware/google/secrets.py#L71.

Reproduction steps: supporting files

No response

Reproduction steps: actual results

The errors seen are as follows:

  File ".../lib/python3.12/site-packages/stormware/google/secrets.py", line 71, in __getitem__
    response = self.client.access_secret_version(name=name)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.12/site-packages/google/cloud/secretmanager_v1/services/secret_manager_service/client.py", line 1875, in access_secret_version
    response = rpc(
               ^^^^
  File ".../lib/python3.12/site-packages/google/api_core/gapic_v1/method.py", line 373, in __call__
    result = wrapped_func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 295, in retry_wrapped_func
    return retry_target(
           ^^^^^^^^^^^^^
  File ".../lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 157, in retry_target
    next_sleep = _retry_error_helper(
                 ^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.12/site-packages/google/api_core/retry/retry_base.py", line 230, in _retry_error_helper
    raise final_exc from source_exc
google.api_core.exceptions.RetryError: Timeout of 60.0s exceeded, last exception: 503 Getting metadata from plugin failed with error: Reauthentication is needed. Please run `gcloud auth application-default login` to reauthenticate.
Reproduction steps: expected results

We'd expect these calls to fail immediately.

OS & version + platform

No response

Python environment

No response

Python dependencies

No response

Additional context

No response

Dominant language
Python
Stars
5.4k
Forks
1.8k
Avg merge
2d 22h
Merged PRs (30d)
102

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from googleapis/google-cloud-python

All issues in googleapis/google-cloud-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.