iiitl / iiitl/Opensource_Compass

Implement Per-User Rate Limiting Middleware in the Core Service

Open
#49 0 comments 0 reactions 0 assignees View on GitHub
backend hard performance security
Dominant language
TypeScript
Stars
0
Forks
16
PR merge metrics
No merged PRs in 30d

Description

### 📋 Description

The Core Service has no rate limiting. Every protected endpoint (recommendations, repo search, setup guide) is guarded by `JWTMiddleware`, which correctly enforces authentication — but **any authenticated user can send unlimited requests**, which is a real problem because:

1. Each request may fan out to the GitHub API and/or the AI/LLM service
2. A misbehaving client or a developer running a test loop can trigger GitHub's secondary rate limits and get the service's shared token blocked
3. There's no fairness guarantee — one user can starve everyone else

The fix is to add a **per-user token bucket** middleware. Each user gets a bucket of N tokens that refills at rate R per second. Each request costs 1 token. If the bucket is empty, the request is rejected with `HTTP 429 Too Many Requests`.

This is a real, production-grade Go task that requires understanding of concurrency, goroutines, `sync.Mutex`, and HTTP middleware chaining.

---

### 📍 Files to Create / Modify

- **Create:** `backend/core_service/internal/ratelimit/limiter.go`
- **Modify:** `backend/core_service/routes/routes.go` — wrap protected routes with the new middleware

---

### 🔍 How Token Bucket Works

```
Each user has a "bucket" with capacity N tokens.
Tokens refill at rate R per second.
Each request consumes 1 token.
If the bucket is empty → reject the request → HTTP 429.
```

Example: `capacity=10, refillRate=2 per second` means a user can burst 10 requests then is limited to 2/s.

---

### ✅ What To Build

**Step 1 — Create `backend/core_service/internal/ratelimit/limiter.go`:**

```go
package ratelimit

import (
"net/http"
"sync"
"time"
)

type bucket struct {
tokens float64
capacity float64
refillRate float64 // tokens per second
lastRefill time.Time
mu sync.Mutex
}

func (b *bucket) allow() bool {
b.mu.Lock()
defer b.mu.Unlock()

now := time.Now()
elapsed := now.Sub(b.lastRefill).Seconds()
b.lastRefill = now

// Refill tokens based on elapsed time
b.tokens += elapsed * b.refillRate
if b.tokens > b.capacity {
b.tokens = b.capacity
}

if b.tokens >= 1.0 {
b.tokens--
return true
}
return false
}

// Limiter holds per-user buckets
type Limiter struct {
mu sync.RWMutex
buckets map[string]*bucket
capacity float64
rate float64
}

func NewLimiter(capacity float64, ratePerSecond float64) *Limiter {
return &Limiter{
buckets: make(map[string]*bucket),
capacity: capacity,
rate: ratePerSecond,
}
}

func (l *Limiter) getBucket(userID string) *bucket {
// Fast path: read lock
l.mu.RLock()
b, ok := l.buckets[userID]
l.mu.RUnlock()
if ok {
return b
}

// Slow path: write lock to create new bucket
l.mu.Lock()
defer l.mu.Unlock()
// Double-check after acquiring write lock
if b, ok = l.buckets[userID]; ok {
return b
}
b = &bucket{
tokens: l.capacity,
capacity: l.capacity,
refillRate: l.rate,
lastRefill: time.Now(),
}
l.buckets[userID] = b
return b
}

// Middleware wraps an http.Handler with per-user rate limiting.
// It reads the authenticated user ID from the request context
// (set by JWTMiddleware using the "user_id" key).
func (l *Limiter) Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
userID, ok := r.Context().Value(contextKey("user_id")).(string)
if !ok || userID == "" {
// Should not happen if JWTMiddleware ran first, but be safe
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}

b := l.getBucket(userID)
if !b.allow() {
w.Header().Set("Retry-After", "1")
http.Error(w, `{"error":"rate limit exceeded, please slow down"}`, http.StatusTooManyRequests)
return
}
next.ServeHTTP(w, r)
})
}

// Must match the key type in auth/middleware.go
type contextKey string
```

**Step 2 — Wire it into `routes/routes.go`:**

```go
// In the route setup function, after creating JWTMiddleware:
limiter := ratelimit.NewLimiter(10, 2) // 10 burst, 2 req/sec sustained

// Example: wrap the recommendations handler
mux.Handle("/recommendations",
auth.JWTMiddleware(jwtSecret,
limiter.Middleware(
http.HandlerFunc(recHandler.GetRecommendation),
),
),
)
```

The middleware chain is: `JWTMiddleware → RateLimitMiddleware → Handler`. This order matters — rate limiting only makes sense for authenticated users (known by user ID).

**Step 3 (Bonus) — Add periodic cleanup to avoid unbounded memory growth:**

If the service runs for weeks, the `buckets` map grows forever (one entry per unique user). Add a cleanup goroutine that removes buckets for users who haven't made a request in a configurable duration (e.g., 1 hour).

---

### 🏁 Acceptance Criteria

- [ ] A `Limiter` struct exists in `ratelimit/limiter.go` using the token bucket algorithm
- [ ] The `bucket.allow()` method is correct: refills on elapsed time, caps at capacity, returns `false` when empty
- [ ] Per-user state is stored safely — bucket creation and token consumption are both protected by mutexes
- [ ] The middleware returns **HTTP 429** with a `Retry-After: 1` header when the limit is exceeded
- [ ] The middleware reads `user_id` from the request context (set by `JWTMiddleware`)
- [ ] At least the `/recommendations` route is wrapped with the rate limiter
- [ ] `go test -race ./internal/ratelimit/...` passes with zero data races

---

### 💡 Technical Hints

- Look at `backend/core_service/internal/auth/middleware.go` for how `user_id` is stored in context using `context.WithValue`. Your middleware reads that same key. **Be careful**: you must use the same `contextKey` type, otherwise `r.Context().Value("user_id")` will return `nil` because Go context keys are compared by both value AND type.
- The **double-checked locking** pattern in `getBucket` (check RLock → check WLock) is important for concurrent correctness and performance. Without it, acquiring a write lock on every request would be a bottleneck.
- Test your rate limiter by writing a `TestRateLimiter` in `limiter_test.go` that fires 15 requests rapidly and asserts that the first 10 succeed and the next 5 are rejected.

---

### 🚀 Getting Started

1. Fork the repository
2. Create a branch: `git checkout -b feat/issue-29-rate-limiting`
3. Create `backend/core_service/internal/ratelimit/limiter.go`
4. Write tests in `backend/core_service/internal/ratelimit/limiter_test.go`
5. Wire into `routes/routes.go`
6. Run: `cd backend/core_service && go test -race ./...`
7. Open a PR with test output showing the rate limiter working!

Contributor guide

Open the contributing guide

Research direction

Start with backend/core_service/internal/auth/middleware.go to verify how user_id is stored in request context, then create the limiter in backend/core_service/internal/ratelimit/limiter.go and wire it in backend/core_service/routes/routes.go. Add limiter_test.go for concurrent and burst behavior, run cd backend/core_service && go test -race ./..., and consider the acceptance criteria met when authenticated recommendations requests are limited per user and excess requests return HTTP 429 with Retry-After: 1.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend, security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.