stacklok / stacklok/mecatl

mecak8s: make Kubernetes Lease renewal capacity explicit and scalable

Open
#1,515 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Problem

The Kubernetes client uses client-go's default 5 QPS / 10 burst limits. Each held session Lease renewal performs a GET followed by an UPDATE, normally every ten seconds. Roughly 25 concurrently held sessions consume the entire steady-state client budget before acquisitions, releases, scheduler election, retries, or other Kubernetes work.

Throttled renewals can miss the 30-second Lease TTL, cancel live work, and trigger avoidable lease-loss recovery.

Evidence

  • internal/app/build.go:newK8sClientset does not set rest.Config.QPS or Burst.
  • internal/adapter/k8slease/k8slease.go:Renew performs GET + UPDATE.
  • cmd/mecak8s/flags.go defaults the Lease TTL to 30 seconds and renewal interval to TTL / 3.

Proposed direction

Define and document a capacity model. Make QPS/Burst explicit and configurable or derive safe values from supported concurrency. Consider reducing renewal round trips or moving high-cardinality session fencing away from the Kubernetes API.

Acceptance criteria

  • Supported concurrent-session capacity is documented.
  • Client throttling settings are explicit and observable.
  • A load test exercises acquisitions and renewals at the supported limit.
  • Sustained supported load does not cause renewal loss or live-run cancellation.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with internal/app/build.go:newK8sClientset, internal/adapter/k8slease/k8slease.go:Renew, and the Lease TTL and renewal defaults in cmd/mecak8s/flags.go. Define and document supported concurrent-session capacity, make throttling settings explicit and observable, and add a load test showing sustained supported load without renewal loss or live-run cancellation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
distributed-systems, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.