GoogleCloudPlatform / GoogleCloudPlatform/cloud-sql-proxy-operator

Bug: Cloud SQL Proxy Operator fails injecting sometimes.

Open
#736 7 comments 3 reactions 2 assignees Claimed by @panavenue View on GitHub
type: bug
Dominant language
Go
Stars
120
Forks
18
PR merge metrics
No merged PRs in 30d

Description

Priority: I would definitely label this issue with the highest priority, since this completely kills off one replica until a pod has been *deleted*

Issue Description:
Sometimes the Cloud SQL Proxy Operator fails to inject the CloudSQL pod. I was not able to successfully get the pod to restart with a regular initContainer (see below) / the operator to inject, unless I really kill the pod.

Current Workaround:
Add `kubectl delete pod` permission to the service account and kill the pod using a `curl` command and a `busybox-curl` image for the init Container.

Impact:
This essentially fully blocks 1 replica: The replica is not able to spawn the pod to connect to the database and hence stays in a completely unusable state.

Killing pods is essentially harmful for pods with bigger images (consider `imagePullPolicy: Always`)

Recommendation:
Introduce / Fix the mechanism that verifies if a pod has really been properly injected with the cloud sql proxy sidecar. E.g. this can be done by checking if port `5432` or whatever DB_PORT is set is reachable. If not reachable / injected kill the pod, so it restarts.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.