googleapis / googleapis/google-cloud-python

App Engine: trace_id label leaks across log records (mutation of lru_cache-cached dict)

Ouverte
#17,321 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
priority: p2 type: bug
Langage dominant
Python
Étoiles
5.4k
Forks
1.8k
Merge moyen
3 j 4 h
PR mergées (30 j)
122

Description

- [x] I determined this is the correct repository in which to report this bug.

## Summary of the issue

**Context**
Running on App Engine (`gae_app`) with `CloudLoggingHandler`, logging multiple records where some carry a trace context (e.g. request-correlated logs) and some do not.

**Expected Behavior:**
The `appengine.googleapis.com/trace_id` label should only be attached to records that actually have a trace, reflecting each record's own trace.

**Actual Behavior:**
Once any record sets the trace label, that value is stored in a process-wide cached dict and leaks into subsequent records — including records that have **no** trace, or a **different** trace. Trace correlation in the App Engine UI becomes wrong (a record gets attributed to a previous record's request/trace).

## API client name and version

google-cloud-logging v3.15.0 (also present on current `main`)

## Reproduction steps: code

```python
import types
from google.cloud.logging_v2.handlers import _monitored_resources as mr

mr._get_environmental_labels.cache_clear() # simulate fresh process
gae = mr.Resource(type=mr._GAE_RESOURCE_TYPE, labels={})

def record(trace):
r = types.SimpleNamespace()
r._trace = trace
return r

print("record 1 (has trace):", dict(mr.add_resource_labels(gae, record("TRACE-1"))))
print("record 2 (no trace) :", dict(mr.add_resource_labels(gae, record(None))))
```

## Reproduction steps: actual results

```
record 1 (has trace): {'appengine.googleapis.com/trace_id': 'TRACE-1'}
record 2 (no trace) : {'appengine.googleapis.com/trace_id': 'TRACE-1'}
```

Record 2 has no trace, yet it carries record 1's `trace_id`. The cached dict itself is permanently polluted for the rest of the process lifetime.

## Reproduction steps: expected results

```
record 1 (has trace): {'appengine.googleapis.com/trace_id': 'TRACE-1'}
record 2 (no trace) : {}
```

## OS & version + platform

App Engine Standard/Flex (`gae_app`); logic bug, reproducible on any platform

## Python environment

Python 3.12

## Additional context

Root cause is in `google/cloud/logging_v2/handlers/_monitored_resources.py`. `_get_environmental_labels` is memoized with `functools.lru_cache`, so it returns the **same dict instance** on every call. `add_resource_labels` then mutates that cached dict in place:

```python
@functools.lru_cache(maxsize=None)
def _get_environmental_labels(resource_type):
...
return labels # cached object

def add_resource_labels(resource, record):
...
labels = _get_environmental_labels(resource.type) # same cached dict each call
if resource.type == _GAE_RESOURCE_TYPE and record._trace is not None:
labels[_GAE_TRACE_ID_LABEL] = record._trace # mutates the cached dict
return labels
```

The per-record trace write persists in the cache and bleeds into later records. A fix is to copy before mutating, e.g. `labels = dict(_get_environmental_labels(resource.type))`.

This only affects the `gae_app` + trace path, but on that path it is a cross-record data correlation bug.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.