googleapis / googleapis/google-cloud-python

google-cloud-storage: BucketMetadataCache (ACO) triggers storage.buckets.get on object-only operations; no supported opt-out

Đang mở
#17,650 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
5.4k
Fork
1.8k
Merge trung bình
3 ngày 4 giờ
Pull request đã merge (30 ngày)
122

Mô tả

## Environment details

- **Programming language**: Python 3.11
- **Package**: `google-cloud-storage`
- **Versions observed**: `3.11.0` and `3.12.0` (the `BucketMetadataCache` / ACO code path is present in both)

## Summary

`Client.__init__` unconditionally instantiates a `BucketMetadataCache` ("App-centric Observability" / ACO). As a side effect, **ordinary object-level operations** (`list_blobs`, `download_*`, `upload_*`, etc.) now trigger a **background `Client.get_bucket()` call**, i.e. the REST `storage.buckets.get` permission — even though the application code never requests bucket metadata and the operation itself only needs `storage.objects.*`.

For principals granted **object-only roles** (`roles/storage.objectViewer`, `roles/storage.objectAdmin`) — which intentionally exclude `storage.buckets.get` — this produces a continuous stream of **denied `storage.buckets.get` entries in Cloud Audit Logs (data_access, severity ERROR)**, one per bucket per client instance.

The failure is swallowed internally (data operations still succeed), so it is not a functional outage — but it is:
- **Alarming, high-volume audit-log noise** (ERROR severity, "lots per second" under fan-out workloads that create many short-lived clients), and
- **Pressure to over-grant IAM** — teams are asked to add `storage.buckets.get` (e.g. via `roles/storage.bucketViewer`/`legacyBucketReader`) purely to satisfy a telemetry probe the application does not use, across *every* bucket the service account ever touches. This directly conflicts with least-privilege.

There is **no supported way to disable it** — no client option and no environment variable (the only observability env var, `ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACES`, is a separate opt-in for OpenTelemetry tracing). The only workaround is mutating the private attribute `client._bucket_metadata_cache = None`, which is fragile and fails open on library upgrades.

## Steps to reproduce

Object-only operations cause a background `get_bucket()` (→ `storage.buckets.get`). The repro below is **fully offline and needs no credentials or real bucket** — it monkeypatches `get_bucket` only to *observe that it is invoked and by whom*:

```python
import time
import traceback

from google.auth.credentials import AnonymousCredentials
from google.cloud import storage

calls = []
def traced_get_bucket(self, bucket_name, *a, **k):
if not calls:
print("get_bucket() caller:\n" + "".join(traceback.format_stack()[-3:-1]))
calls.append(bucket_name)
raise RuntimeError("short-circuit (offline repro)") # swallowed by ACO bg thread
storage.Client.get_bucket = traced_get_bucket

client = storage.Client(project="proof", credentials=AnonymousCredentials())
# Optional workaround under test: client._bucket_metadata_cache = None
try:
list(client.list_blobs("some-bucket", prefix="x/", max_results=1))
except Exception:
pass # objects.list fails offline; irrelevant to the point
time.sleep(1.0) # let the ACO daemon thread run

print(f"google-cloud-storage {storage.__version__}: "
f"get_bucket invoked {len(calls)} time(s) by one list_blobs() call")
```

### Actual output

```
get_bucket() caller:
File ".../google/cloud/storage/_bucket_metadata_cache.py", line 98, in _fetch_background
bucket = self._client.get_bucket(bucket_name, timeout=10.0)
google-cloud-storage 3.11.0: get_bucket invoked 1 time(s) by one list_blobs() call
```

With `client._bucket_metadata_cache = None` uncommented, the count is `0`.

### Root cause (call chain)

1. `_helpers.create_trace_span_helper` wraps every operation. For any span **not** in `{Storage.Client.getBucket, Storage.Client.lookupBucket, Storage.Bucket.reload, Storage.Bucket.exists}` (i.e. object ops like `listBlobs`), it calls `client._bucket_metadata_cache.get_or_queue_fetch(bucket_name)`.
2. On a cache miss, `get_or_queue_fetch` spawns a daemon thread → `_bucket_metadata_cache._fetch_background`.
3. `_fetch_background` calls `self._client.get_bucket(bucket_name, timeout=10.0)` → REST `storage.buckets.get`.
4. On `Forbidden`, it caches a fallback and returns; the object op proceeds. Every fresh client re-probes on first touch of each bucket.

## Expected behavior

Object-level operations should not require `storage.buckets.get`. This is a long-standing, explicitly-recognized contract for this library — cf. googleapis/python-storage #528 ("Redesign of API so it does not need `storage.buckets.get` access for every single operation") and #1225 ("Need to be able to download objects without `storage.buckets.get` permission same as gsutil"). `gsutil`/`gcloud storage` operate on objects without `buckets.get`, and object-only IAM roles deliberately omit it.

At minimum, ACO's background metadata probe should be **opt-out via a supported mechanism**, and ideally should not fire for principals/operations that don't need bucket metadata.

## Requested change

A **supported opt-out**, e.g. one or more of:

1. A `Client` constructor option (e.g. `enable_bucket_metadata_cache: bool = True`), and/or
2. An environment variable mirroring the existing `ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACES` pattern (e.g. `DISABLE_GCS_PYTHON_CLIENT_BUCKET_METADATA_CACHE`), and/or
3. Making the probe lazy/suppressible so a `Forbidden` result is cached **once per client** without re-emitting audit-log errors, and documenting that ACO requires `storage.buckets.get`.

Any of these lets callers using object-only roles avoid the audit-log noise **without granting `storage.buckets.get`**, and gives a stable alternative to mutating the private `_bucket_metadata_cache` attribute.

Happy to open a PR for option 1 or 2 if a maintainer confirms the preferred shape.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start with Client.__init__, _helpers.create_trace_span_helper, and google/cloud/storage/_bucket_metadata_cache.py to trace when object operations queue metadata fetches. Compare the requested opt-out approaches, then add coverage showing the selected supported mechanism prevents the background get_bucket call while object operations remain usable.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
python
Lĩnh vực
api, cloud
Loại issue
Tính năng
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Sôi nổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
52/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.