iiitl / iiitl/Opensource_Compass
Add TTL Caching to the Setup Guide Handler (Stop Calling the LLM on Every Visit)
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
### π Description
Every time a user visits `/repo/[owner]/[repo]/setup`, the frontend calls the Core Service `GET /repo/setup-guide?repo=owner/repo&user_id=...`. The handler in `setup_guide.go` always:
1. Fetches the repo's README from the GitHub API
2. Makes a full LLM call to the AI service to generate the guide
There is **zero caching**. This means:
- A user who clicks "Setup Guide" and then navigates away and comes back triggers **two full LLM calls**
- Multiple users with the same experience level viewing the same popular repo each trigger individual LLM calls
- LLM API calls cost money and have their own rate limits β this design wastes both
The fix is to build a `SetupGuideCache` that caches responses keyed by `(owner, repo, experience_level)` with a TTL. Since README content changes rarely, a TTL of several hours is appropriate.
This is harder than the generic cache cleanup issue (#18 ) because:
- The cache key must combine 3 fields
- The cached value is a complex JSON struct (`SetupGuide`), not a simple string
- You must integrate it cleanly into an existing HTTP handler without breaking the existing code path
---
### π Files to Create / Modify
- **Create:** `backend/core_service/internal/cache/setup_guide_cache.go`
- **Modify:** `backend/core_service/internal/handlers/setup_guide.go` β check cache before calling AI, write to cache after
---
### π Current Handler (Always Calls AI)
```go
// setup_guide.go
func (h *SetupGuideHandler) GetSetupGuide(w http.ResponseWriter, r *http.Request) {
// ... parameter validation ...
readme, _ := h.githubClient.FetchReadme(ctx, owner, repoName, token) // GitHub call
guide, _ := h.aiClient.GetSetupGuide(ctx, readme, user.ExperienceLevel) // LLM call β always happens
json.NewEncoder(w).Encode(map[string]string{"guide": guide})
}
```
---
### β What To Build
**Step 1 β Create `backend/core_service/internal/cache/setup_guide_cache.go`:**
```go
package cache
import (
"fmt"
"sync"
"time"
)
type SetupGuideCacheEntry struct {
GuideJSON string // The raw JSON string from the AI
CachedAt time.Time
}
type SetupGuideCache struct {
mu sync.RWMutex
entries map[string]SetupGuideCacheEntry
ttl time.Duration
}
func NewSetupGuideCache(ttl time.Duration) *SetupGuideCache {
c := &SetupGuideCache{
entries: make(map[string]SetupGuideCacheEntry),
ttl: ttl,
}
// Periodically clean up expired entries
go c.startCleanup(ttl * 2)
return c
}
// Key combines owner, repo, and experience level to ensure per-user-type caching
func (c *SetupGuideCache) cacheKey(owner, repo, experienceLevel string) string {
return fmt.Sprintf("%s/%s::%s", owner, repo, experienceLevel)
}
func (c *SetupGuideCache) Get(owner, repo, experienceLevel string) (string, bool) {
key := c.cacheKey(owner, repo, experienceLevel)
c.mu.RLock()
entry, ok := c.entries[key]
c.mu.RUnlock()
if !ok || time.Since(entry.CachedAt) > c.ttl {
return "", false
}
return entry.GuideJSON, true
}
func (c *SetupGuideCache) Set(owner, repo, experienceLevel, guideJSON string) {
key := c.cacheKey(owner, repo, experienceLevel)
c.mu.Lock()
c.entries[key] = SetupGuideCacheEntry{
GuideJSON: guideJSON,
CachedAt: time.Now(),
}
c.mu.Unlock()
}
func (c *SetupGuideCache) startCleanup(interval time.Duration) {
ticker := time.NewTicker(interval)
defer ticker.Stop()
for range ticker.C {
c.mu.Lock()
now := time.Now()
for k, v := range c.entries {
if now.Sub(v.CachedAt) > c.ttl {
delete(c.entries, k)
}
}
c.mu.Unlock()
}
}
```
**Step 2 β Add cache to `SetupGuideHandler` struct and constructor:**
```go
// In setup_guide.go:
type SetupGuideHandler struct {
githubClient *clients.GitHubClient
aiClient *clients.AIClient
userRepo *users.Repository
cache *cache.SetupGuideCache // β add this
}
func NewSetupGuideHandler(gh *clients.GitHubClient, ai *clients.AIClient, ur *users.Repository, c *cache.SetupGuideCache) *SetupGuideHandler {
return &SetupGuideHandler{githubClient: gh, aiClient: ai, userRepo: ur, cache: c}
}
```
**Step 3 β Check cache before calling AI, populate cache after:**
```go
func (h *SetupGuideHandler) GetSetupGuide(w http.ResponseWriter, r *http.Request) {
// ... existing parameter validation ...
// βββ Cache Check ββββββββββββββββββββββββββββββββββββ
if h.cache != nil {
if cached, ok := h.cache.Get(owner, repoName, user.ExperienceLevel); ok {
log.Printf("SetupGuide: cache HIT for %s/%s (%s)", owner, repoName, user.ExperienceLevel)
w.Header().Set("Content-Type", "application/json")
w.Header().Set("X-Cache", "HIT") // helpful for debugging
w.Write([]byte(`{"guide":` + cached + `}`))
return
}
log.Printf("SetupGuide: cache MISS for %s/%s (%s)", owner, repoName, user.ExperienceLevel)
}
// βββ Existing logic (fetch readme + call AI) βββββββββ
readme, err := h.githubClient.FetchReadme(r.Context(), owner, repoName, user.GitHubToken)
if err != nil {
http.Error(w, "failed to fetch readme: "+err.Error(), http.StatusInternalServerError)
return
}
guide, err := h.aiClient.GetSetupGuide(r.Context(), readme, user.ExperienceLevel)
if err != nil {
http.Error(w, "failed to generate guide: "+err.Error(), http.StatusInternalServerError)
return
}
// βββ Store in cache βββββββββββββββββββββββββββββββββββ
if h.cache != nil {
h.cache.Set(owner, repoName, user.ExperienceLevel, guide)
}
w.Header().Set("Content-Type", "application/json")
w.Header().Set("X-Cache", "MISS")
json.NewEncoder(w).Encode(map[string]string{"guide": guide})
}
```
**Step 4 β Wire it up in `main.go` or `routes.go`:**
```go
guideCache := cache.NewSetupGuideCache(4 * time.Hour) // cached for 4 hours
setupHandler := handlers.NewSetupGuideHandler(githubClient, aiClient, userRepo, guideCache)
```
---
### π Acceptance Criteria
- [ ] A `SetupGuideCache` with `Get`, `Set`, and background cleanup is implemented
- [ ] Cache key includes `owner`, `repo`, and `experience_level` β two users with different levels get different cached responses
- [ ] On a cache HIT, the handler returns immediately without calling the AI service
- [ ] The response includes an `X-Cache: HIT` or `X-Cache: MISS` header (useful for observability)
- [ ] The TTL is configurable β passed to `NewSetupGuideCache()` as a parameter, not hardcoded
- [ ] Background cleanup goroutine removes expired entries to prevent unbounded memory growth
- [ ] All code compiles: `cd backend/core_service && go build ./...`
- [ ] Verify manually: hit the setup guide endpoint twice for the same repo, confirm the second response is significantly faster (< 10ms vs potentially seconds)
---
### π‘ Technical Hints
- The AI service returns a guide as a JSON string (look at `AIClient.GetSetupGuide` return type). Cache the **raw JSON string** β don't unmarshal and re-marshal it, that wastes CPU on every cache write/read
- `experienceLevel` is part of the cache key because a "beginner" and an "intermediate" user should get differently-worded guides for the same repo
- The handler currently updates `NewSetupGuideHandler` call β find where it's instantiated (likely `main.go` or `routes/`) and update that call to pass the cache
---
### π Getting Started
1. Fork the repository
2. Create a branch: `git checkout -b feat/issue-32-setup-guide-cache`
3. Create `backend/core_service/internal/cache/setup_guide_cache.go`
4. Update `setup_guide.go` handler
5. Update constructor call in `main.go`/`routes/`
6. Test: curl the same endpoint twice, 2nd response should be nearly instant
7. Open a PR with `curl` output or logs showing MISS then HIT!
Contributor guide
Research direction
Start with backend/core_service/internal/handlers/setup_guide.go and inspect the SetupGuideHandler constructor and endpoint flow, then create backend/core_service/internal/cache/setup_guide_cache.go. Trace the constructor call in main.go or routes/ and run cd backend/core_service && go build ./.... Done means repeated requests use the configured TTL cache, include the expected X-Cache header, and show MISS then HIT behavior for the same repository and experience level.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 70/100