zeroae / zeroae/zae-limiter

✨ CloudWatch metrics for ConfigCache (hit rate, cold/warm latency)

Open
#322 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/limiter performance
Dominant language
Python
Stars
0
Forks
0
Avg merge
6h 51m
Merged PRs (30d)
104

Description

Problem or Use Case

The ConfigCache already tracks hits and misses internally via CacheStats, but these metrics are only available programmatically through get_cache_stats(). There is no integration with CloudWatch, so operators cannot:

  • Monitor cache hit rates in production dashboards
  • Measure cold-cache vs warm-cache latency (p50/p95/p99)
  • Set alarms on cache degradation (e.g., hit rate drops below threshold)
  • Validate the effectiveness of the BatchGetItem optimization (#298) in production

Split from #298 to keep the BatchGetItem optimization focused on DynamoDB access patterns while this issue addresses observability.

Proposed Solution

Custom CloudWatch Metrics

Publish metrics from ConfigCache to CloudWatch using the existing boto3/aioboto3 clients:

Metric Name Unit Description
ConfigCache/HitRate Percent hits / (hits + misses) over reporting period
ConfigCache/Hits Count Cache hits since last publish
ConfigCache/Misses Count Cache misses since last publish
ConfigCache/ColdCacheLatency Milliseconds Latency on cache miss (includes DynamoDB round trip)
ConfigCache/WarmCacheLatency Milliseconds Latency on cache hit (in-memory lookup)
ConfigCache/Size Count Number of cached entries

Dimensions: StackName (required), Resource (optional, for per-resource breakdown)

Integration Points
  1. CacheStats already tracks hits/misses - extend with latency tracking (start/stop timers around fetch_fn calls)
  2. Periodic publishing - batch metrics and publish at configurable intervals (e.g., every 60s) to avoid per-request CloudWatch API calls
  3. Opt-in - disabled by default, enabled via enable_metrics=True on RateLimiter or ConfigCache
Latency Tracking
# On cache miss: measure fetch_fn latency
start = time.monotonic()
value = await fetch_fn()
elapsed_ms = (time.monotonic() - start) * 1000
self._cold_latencies.append(elapsed_ms)

# On cache hit: measure lookup latency
start = time.monotonic()
value = entry.value  # in-memory
elapsed_ms = (time.monotonic() - start) * 1000
self._warm_latencies.append(elapsed_ms)

Acceptance Criteria

  • CacheStats extended with cold_latency_ms and warm_latency_ms lists for percentile calculation
  • New ConfigCacheMetrics class (or equivalent) publishes custom metrics to CloudWatch namespace ZaeLimiter/ConfigCache
  • Metrics include HitRate, Hits, Misses, ColdCacheLatency, WarmCacheLatency, and Size
  • All metrics tagged with StackName dimension; latency metrics support optional Resource dimension
  • Metrics publishing is opt-in (disabled by default), enabled via a RateLimiter constructor parameter
  • Metrics are batched and published periodically (not per-request) to minimize CloudWatch API costs
  • Unit tests in tests/unit/ verify metric values match CacheStats counters
  • Unit tests verify latency is recorded on cache hit and cache miss code paths
  • Sync variant generated via generate_sync.py if new async source files are added

Alternatives Considered

  1. EMF (Embedded Metrics Format) via Lambda Powertools: Only works inside Lambda. ConfigCache runs in the application process, not the aggregator Lambda.

  2. Expose Prometheus endpoint: Adds a dependency and requires a metrics scraper. CloudWatch is already available in the AWS environment.

  3. Log-based metrics (CloudWatch Logs Insights): Requires structured logging and post-hoc queries. Custom metrics provide real-time dashboards and alarms.

Dependencies

  • #298 - BatchGetItem optimization (cold-cache latency is what this issue measures)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the existing ConfigCache, CacheStats, and RateLimiter implementations first, then inspect tests/unit/ and generate_sync.py. Run the relevant unit tests and trace the cache hit and miss paths before assessing the metrics design. Done means opt-in, batched CloudWatch metrics cover the listed counters, latency, dimensions, and cache size, with tests verifying the values.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
backend, cloud, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.