googleapis / googleapis/google-cloud-python
App Engine: trace_id label leaks across log records (mutation of lru_cache-cached dict)
- Lenguaje dominante
- Python
- Estrellas
- 5.4k
- Forks
- 1.8k
- Merge medio
- 3 d 4 h
- PR fusionados (30 d)
- 122
Descripción
- [x] I determined this is the correct repository in which to report this bug.
## Summary of the issue
**Context**
Running on App Engine (`gae_app`) with `CloudLoggingHandler`, logging multiple records where some carry a trace context (e.g. request-correlated logs) and some do not.
**Expected Behavior:**
The `appengine.googleapis.com/trace_id` label should only be attached to records that actually have a trace, reflecting each record's own trace.
**Actual Behavior:**
Once any record sets the trace label, that value is stored in a process-wide cached dict and leaks into subsequent records — including records that have **no** trace, or a **different** trace. Trace correlation in the App Engine UI becomes wrong (a record gets attributed to a previous record's request/trace).
## API client name and version
google-cloud-logging v3.15.0 (also present on current `main`)
## Reproduction steps: code
```python
import types
from google.cloud.logging_v2.handlers import _monitored_resources as mr
mr._get_environmental_labels.cache_clear() # simulate fresh process
gae = mr.Resource(type=mr._GAE_RESOURCE_TYPE, labels={})
def record(trace):
r = types.SimpleNamespace()
r._trace = trace
return r
print("record 1 (has trace):", dict(mr.add_resource_labels(gae, record("TRACE-1"))))
print("record 2 (no trace) :", dict(mr.add_resource_labels(gae, record(None))))
```
## Reproduction steps: actual results
```
record 1 (has trace): {'appengine.googleapis.com/trace_id': 'TRACE-1'}
record 2 (no trace) : {'appengine.googleapis.com/trace_id': 'TRACE-1'}
```
Record 2 has no trace, yet it carries record 1's `trace_id`. The cached dict itself is permanently polluted for the rest of the process lifetime.
## Reproduction steps: expected results
```
record 1 (has trace): {'appengine.googleapis.com/trace_id': 'TRACE-1'}
record 2 (no trace) : {}
```
## OS & version + platform
App Engine Standard/Flex (`gae_app`); logic bug, reproducible on any platform
## Python environment
Python 3.12
## Additional context
Root cause is in `google/cloud/logging_v2/handlers/_monitored_resources.py`. `_get_environmental_labels` is memoized with `functools.lru_cache`, so it returns the **same dict instance** on every call. `add_resource_labels` then mutates that cached dict in place:
```python
@functools.lru_cache(maxsize=None)
def _get_environmental_labels(resource_type):
...
return labels # cached object
def add_resource_labels(resource, record):
...
labels = _get_environmental_labels(resource.type) # same cached dict each call
if resource.type == _GAE_RESOURCE_TYPE and record._trace is not None:
labels[_GAE_TRACE_ID_LABEL] = record._trace # mutates the cached dict
return labels
```
The per-record trace write persists in the cache and bleeds into later records. A fix is to copy before mutating, e.g. `labels = dict(_get_environmental_labels(resource.type))`.
This only affects the `gae_app` + trace path, but on that path it is a cross-record data correlation bug.
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.