googleapis / googleapis/google-cloud-python

App Engine: trace_id label leaks across log records (mutation of lru_cache-cached dict)

Đang mở
#17,321 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
priority: p2 type: bug
Ngôn ngữ chính
Python
Star
5.4k
Fork
1.8k
Merge trung bình
3 ngày 4 giờ
Pull request đã merge (30 ngày)
122

Mô tả

- [x] I determined this is the correct repository in which to report this bug.

## Summary of the issue

**Context**
Running on App Engine (`gae_app`) with `CloudLoggingHandler`, logging multiple records where some carry a trace context (e.g. request-correlated logs) and some do not.

**Expected Behavior:**
The `appengine.googleapis.com/trace_id` label should only be attached to records that actually have a trace, reflecting each record's own trace.

**Actual Behavior:**
Once any record sets the trace label, that value is stored in a process-wide cached dict and leaks into subsequent records — including records that have **no** trace, or a **different** trace. Trace correlation in the App Engine UI becomes wrong (a record gets attributed to a previous record's request/trace).

## API client name and version

google-cloud-logging v3.15.0 (also present on current `main`)

## Reproduction steps: code

```python
import types
from google.cloud.logging_v2.handlers import _monitored_resources as mr

mr._get_environmental_labels.cache_clear() # simulate fresh process
gae = mr.Resource(type=mr._GAE_RESOURCE_TYPE, labels={})

def record(trace):
r = types.SimpleNamespace()
r._trace = trace
return r

print("record 1 (has trace):", dict(mr.add_resource_labels(gae, record("TRACE-1"))))
print("record 2 (no trace) :", dict(mr.add_resource_labels(gae, record(None))))
```

## Reproduction steps: actual results

```
record 1 (has trace): {'appengine.googleapis.com/trace_id': 'TRACE-1'}
record 2 (no trace) : {'appengine.googleapis.com/trace_id': 'TRACE-1'}
```

Record 2 has no trace, yet it carries record 1's `trace_id`. The cached dict itself is permanently polluted for the rest of the process lifetime.

## Reproduction steps: expected results

```
record 1 (has trace): {'appengine.googleapis.com/trace_id': 'TRACE-1'}
record 2 (no trace) : {}
```

## OS & version + platform

App Engine Standard/Flex (`gae_app`); logic bug, reproducible on any platform

## Python environment

Python 3.12

## Additional context

Root cause is in `google/cloud/logging_v2/handlers/_monitored_resources.py`. `_get_environmental_labels` is memoized with `functools.lru_cache`, so it returns the **same dict instance** on every call. `add_resource_labels` then mutates that cached dict in place:

```python
@functools.lru_cache(maxsize=None)
def _get_environmental_labels(resource_type):
...
return labels # cached object

def add_resource_labels(resource, record):
...
labels = _get_environmental_labels(resource.type) # same cached dict each call
if resource.type == _GAE_RESOURCE_TYPE and record._trace is not None:
labels[_GAE_TRACE_ID_LABEL] = record._trace # mutates the cached dict
return labels
```

The per-record trace write persists in the cache and bleeds into later records. A fix is to copy before mutating, e.g. `labels = dict(_get_environmental_labels(resource.type))`.

This only affects the `gae_app` + trace path, but on that path it is a cross-record data correlation bug.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.