zalando / zalando/postgres-operator

Pod Deletion Event Not Caught by waitForPodDeletion Function

Open
#2,919 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
5.2k
Forks
1.1k
Avg merge
2d 16h
Merged PRs (30d)
3

Description

Environment Information

  • Which image of the operator are you using? master branch
  • Where do you run it - cloud or metal? Kubernetes or OpenShift? Bare Metal K8s
  • Are you running Postgres Operator in production? No (test environment)
  • Type of issue? Bug report

File Path

pkg/cluster/pod.go in the recreatePod function

Description of the Issue

Sometimes, when a pod has already been deleted, the waitForPodDeletion function in the recreatePod method fails to catch the PodDeletion Event. This causes the function to hang or timeout, even though the pod deletion has actually occurred.

Steps to Reproduce

  1. Trigger pod recreation via the recreatePod function
  2. In some cases, the pod gets deleted successfully
  3. However, the waitForPodDeletion function doesn't detect this event
  4. The process gets stuck waiting for an event that won't come

Expected Behavior

The waitForPodDeletion function should reliably detect when a pod has been deleted, regardless of timing or race conditions.

Actual Behavior

The function sometimes misses the deletion event, causing the process to hang.

Additional Information

This issue appears to be related to event handling in the operator. It might be a race condition where the pod deletion event occurs before the event listener is properly set up, or the event is somehow missed by the subscriber mechanism.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in pkg/cluster/pod.go at the recreatePod function and trace how waitForPodDeletion subscribes to and handles pod deletion events. Reproduce the timing where deletion precedes listener setup, then verify that deletion is reliably observed and the function no longer hangs or times out after the pod is gone.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes, postgresql
Domain
infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.