spiffe / spiffe/spire-controller-manager

feature request: add in-memory cache for entry list

Open
#705 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
77
Forks
69
Avg merge
11d 11h
Merged PRs (30d)
6

Description

Problem

When running large clusters, calling ListEntries on every reconcile is not really practical, especially when multiple controller managers are connected to a single SPIRE Server cluster.

A nested SPIRE Server setup could be one possible solution, but adding a cache mechanism to spire-controller-manager seems much simpler.
(#663 related to this maybe?)

for context, our largest Kubernetes cluster has around 30k pods in a single cluster, and roughly 300k–500k entries are registered in SPIRE Server in total.

Proposed Solution

  • implement in-memory caching mechanism to avoid listing all entries when reconciling
  • these features might work well if it's used together
    • entry prefixing #325
    • hint field based filtering (no issue/PR but spire server support this and I use this in my forked code)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the ListEntries calls made during reconciliation in spire-controller-manager and review the related entry-prefixing discussion in #325. Compare the proposed cache with the entry-listing and filtering needs described here, including the possible hint field. Done means reconciliation no longer lists all entries on every cycle while retaining correct entry results.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.