googleapis / googleapis/google-cloud-python
InvalidArgument error: "resource labels is incomplete, missing (instance_id)" when exporting built-in metrics
- 主要语言
- Python
- 星标
- 5.4k
- 派生
- 1.8k
- 平均合并
- 2 天 23 小时
- 30 天内合并 PR
- 123
描述
After upgrading to `google-cloud-spanner` v3.60.0+, the built-in OpenTelemetry metrics feature (enabled by default) fails to export metrics to Cloud Monitoring with the following error:
```text
InvalidArgument: One or more TimeSeries could not be written:
timeSeries[N] (metric.type="spanner.googleapis.com/internal/client/operation_latencies", ...):
the set of resource labels is incomplete, missing (instance_id)
```
**Impact**: This error does not affect Spanner operations (queries succeed with HTTP 200), but it:
- Generates significant log noise
- Floods error tracking systems (e.g., Sentry) with thousands of errors
- Prevents built-in metrics from being collected in Cloud Monitoring
---
## Environment
- **google-cloud-spanner version**: 3.62.0 (also affects 3.60.0, 3.61.0)
- **Python version**: 3.13.8
- **OS**: Linux (Cloud Run)
- **Previous working version**: 3.59.x and earlier (metrics were opt-in)
---
## Minimal Reproduction
```python
from google.cloud import spanner
# With google-cloud-spanner >= 3.60.0, built-in metrics are enabled by default
client = spanner.Client(project="my-project")
instance = client.instance("my-instance")
database = instance.database("my-database")
with database.snapshot() as snapshot:
results = snapshot.execute_sql("SELECT 1")
list(results) # Execute query
# After ~60 seconds (metrics export interval), the following error appears in logs:
# google.api_core.exceptions.InvalidArgument: ... missing (instance_id)
```
---
## Expected Behavior
All required resource labels, including `instance_id`, are set, and the error does not occur.
---
## Actual Behavior
Metrics export fails with `InvalidArgument` error. The resource labels in the error message show:
```text
resource.labels={"location": "asia-northeast1", "instance_config": "unknown", "client_hash": "XXXXXX"}
```
## Stack Trace
```text
File "opentelemetry/sdk/metrics/_internal/export/__init__.py", line 550, in _receive_metrics
self._exporter.export(
File "google/cloud/spanner_v1/metrics/metrics_exporter.py", line 357, in export
self._batch_write(time_series_list, timeout_millis)
File "google/cloud/spanner_v1/metrics/metrics_exporter.py", line 154, in _batch_write
self.client.create_service_time_series(
File "google/cloud/monitoring_v3/services/metric_service/client.py", line 1937, in create_service_time_series
rpc(
File "google/api_core/gapic_v1/method.py", line 131, in __call__
return wrapped_func(*args, **kwargs)
File "google/api_core/grpc_helpers.py", line 77, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: One or more TimeSeries could not be written:
... the set of resource labels is incomplete, missing (instance_id)
```
贡献指南
评估
这个 Issue 还没有评估数据。