kelos-dev / kelos-dev/kelos

ETag transport cache grows unbounded in long-running spawner

Open
#683 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

actor/kelos kind/cleanup priority/important-soon triage-accepted
Dominant language
Go
Stars
331
Forks
40
Avg merge
1d 21h
Merged PRs (30d)
70

Description

What happened?

The etagTransport introduced in #682 caches every unique GET URL (including paginated URLs like ?page=2&per_page=100) for the lifetime of the process. In a long-running taskspawner-loop controller, this map grows without bound and leaks memory over days/weeks.

What did you expect to happen?

The cache should have a bounded size or TTL-based eviction so memory usage stays stable in long-running deployments.

Possible approaches
  • Cap the cache at a fixed number of entries (e.g., 1000) with simple LRU eviction
  • Add a TTL per entry (e.g., 10 minutes) so stale entries are reaped
  • Both — LRU + TTL
Additional context

In the typical single-repo case, the number of unique URLs per cycle is small (issues list pages + comment pages), so this is low-urgency. It becomes more relevant if a spawner targets a high-volume repo with many paginated comment threads.

Introduced in #682.

/kind cleanup

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the etagTransport implementation introduced in issue #682 and trace how the taskspawner-loop creates and reuses it across requests. Compare the cache lifetime with the loop lifetime, then define and verify an eviction policy that keeps memory bounded during long-running, paginated workloads.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.