GoogleCloudPlatform / GoogleCloudPlatform/cloud-sql-proxy

Intermittent errors logged after enabling telemetry

Offen
#2,018 10 Kommentare 1 Reaktion 1 zugewiesene Person Zugewiesen an @hessjcg Auf GitHub ansehen
priority: p3 type: bug
Vorherrschende Sprache
Go
Sterne
1.4k
Forks
352
Ø Merge
14 Std. 54 Min.
Gemergte PRs (30 T.)
5

Beschreibung

### Bug Description

We are running gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.7.0 as a container next to our main http api container for connectivity to our CloudSQL instance.

After enabling telemetry using the `--telemetry-project` and `-telemetry-prefix` flags we have recurrently gotten the following error logged:

`2023/11/04 13:58:43 Failed to export to Stackdriver: rpc error: code = Internal desc = One or more TimeSeries could not be written: Internal error encountered. Please retry after a few seconds. If internal errors persist, contact support at https://cloud.google.com/support/docs.: global{} timeSeries[0]: custom.googleapis.com/opencensus/_cloud_sql_proxy/cloudsqlconn/refresh_success_count{opencensus_task:go-1@,cloudsql_instance:}; Internal error encountered. Please retry after a few seconds. If internal errors persist, contact support at https://cloud.google.com/support/docs.: global{} timeSeries[1]: custom.googleapis.com/opencensus/_cloud_sql_proxy/cloudsqlconn/dial_latency{cloudsql_instance:,opencensus_task:go-1@}`

However when expecting the metrics we can see that it works as expected. So this is mostly causes the issue of polluted logs. But it would also be interesting to understand why this error is reported.

### Example code (or command)

```
// paste your code or command here
```

### Stacktrace

```bash
`2023/11/04 13:58:43 Failed to export to Stackdriver: rpc error: code = Internal desc = One or more TimeSeries could not be written: Internal error encountered. Please retry after a few seconds. If internal errors persist, contact support at https://cloud.google.com/support/docs.: global{} timeSeries[0]: custom.googleapis.com/opencensus/_cloud_sql_proxy/cloudsqlconn/refresh_success_count{opencensus_task:go-1@,cloudsql_instance:}; Internal error encountered. Please retry after a few seconds. If internal errors persist, contact support at https://cloud.google.com/support/docs.: global{} timeSeries[1]: custom.googleapis.com/opencensus/_cloud_sql_proxy/cloudsqlconn/dial_latency{cloudsql_instance:,opencensus_task:go-1@}`
```

### Steps to reproduce?

1. Launch cloud-sql-proxy 2.7.0 as a container in GCP GKE.
2. Inspect logs.

### Environment

1. OS type and version: GCP GKE 1.24.14-gke.2700
2. Cloud SQL Proxy version: 2.7.0
3. Proxy invocation command: `apiVersion: apps/v1
kind: Deployment
metadata:
name:
spec:
template:
spec:
containers:
- name: cloudsql-proxy
image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.7.0
args:
- "--auto-iam-authn"
- "--max-sigterm-delay"
- "25s"
- "--structured-logs"
- "--telemetry-project"
- ""
- "--telemetry-prefix"
- "_cloud_sql_proxy"
- ""
`

### Additional Details

_No response_

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.