mittwald / mittwald/kube-httpcache

Impossible to cluster when having readiness gates on port 8080

Open
#222 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Go
Stars
313
Forks
82
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
When using clustering in combination with Readiness Gates (AWS ALB readiness gates), it is impossible to start the pods, because no endpoint will become available until the endpoints have been populated, but the endpoints will never be populated until the readiness gate passes. This ends up in a loop which never allows a pod to be fully started.

To Reproduce
Steps to reproduce the behavior:

  1. Use EKS
  2. Deploy kube-httpcache, 2 pods minimum
  3. Look at the logs from kube httpcache, an error message like the following one appears:
W0308 14:41:30.853956       1 endpoints_watch.go:66] service 'some_random_service' has no endpoints

Expected behavior
I would expect to be able to cluster the pods.

Environment:

  • Kubernetes version: [e.g. 1.26]
  • kube-httpcache version: [e.g. v0.7]

Configuration

Additional context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing endpoints_watch.go around line 66 and reproduce the two-pod setup on EKS with AWS ALB readiness gates. Compare the endpoint population and readiness behavior, then verify that the pods can start clustering without remaining in the no-endpoints loop.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go, kubernetes
Domain
infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.