iiitl / iiitl/Opensource_Compass
Implement Per-User Rate Limiting Middleware in the Core Service
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
### 📋 Description
The Core Service has no rate limiting. Every protected endpoint (recommendations, repo search, setup guide) is guarded by `JWTMiddleware`, which correctly enforces authentication — but **any authenticated user can send unlimited requests**, which is a real problem because:
1. Each request may fan out to the GitHub API and/or the AI/LLM service
2. A misbehaving client or a developer running a test loop can trigger GitHub's secondary rate limits and get the service's shared token blocked
3. There's no fairness guarantee — one user can starve everyone else
The fix is to add a **per-user token bucket** middleware. Each user gets a bucket of N tokens that refills at rate R per second. Each request costs 1 token. If the bucket is empty, the request is rejected with `HTTP 429 Too Many Requests`.
This is a real, production-grade Go task that requires understanding of concurrency, goroutines, `sync.Mutex`, and HTTP middleware chaining.
---
### 📍 Files to Create / Modify
- **Create:** `backend/core_service/internal/ratelimit/limiter.go`
- **Modify:** `backend/core_service/routes/routes.go` — wrap protected routes with the new middleware
---
### 🔍 How Token Bucket Works
```
Each user has a "bucket" with capacity N tokens.
Tokens refill at rate R per second.
Each request consumes 1 token.
If the bucket is empty → reject the request → HTTP 429.
```
Example: `capacity=10, refillRate=2 per second` means a user can burst 10 requests then is limited to 2/s.
---
### ✅ What To Build
**Step 1 — Create `backend/core_service/internal/ratelimit/limiter.go`:**
```go
package ratelimit
import (
"net/http"
"sync"
"time"
)
type bucket struct {
tokens float64
capacity float64
refillRate float64 // tokens per second
lastRefill time.Time
mu sync.Mutex
}
func (b *bucket) allow() bool {
b.mu.Lock()
defer b.mu.Unlock()
now := time.Now()
elapsed := now.Sub(b.lastRefill).Seconds()
b.lastRefill = now
// Refill tokens based on elapsed time
b.tokens += elapsed * b.refillRate
if b.tokens > b.capacity {
b.tokens = b.capacity
}
if b.tokens >= 1.0 {
b.tokens--
return true
}
return false
}
// Limiter holds per-user buckets
type Limiter struct {
mu sync.RWMutex
buckets map[string]*bucket
capacity float64
rate float64
}
func NewLimiter(capacity float64, ratePerSecond float64) *Limiter {
return &Limiter{
buckets: make(map[string]*bucket),
capacity: capacity,
rate: ratePerSecond,
}
}
func (l *Limiter) getBucket(userID string) *bucket {
// Fast path: read lock
l.mu.RLock()
b, ok := l.buckets[userID]
l.mu.RUnlock()
if ok {
return b
}
// Slow path: write lock to create new bucket
l.mu.Lock()
defer l.mu.Unlock()
// Double-check after acquiring write lock
if b, ok = l.buckets[userID]; ok {
return b
}
b = &bucket{
tokens: l.capacity,
capacity: l.capacity,
refillRate: l.rate,
lastRefill: time.Now(),
}
l.buckets[userID] = b
return b
}
// Middleware wraps an http.Handler with per-user rate limiting.
// It reads the authenticated user ID from the request context
// (set by JWTMiddleware using the "user_id" key).
func (l *Limiter) Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
userID, ok := r.Context().Value(contextKey("user_id")).(string)
if !ok || userID == "" {
// Should not happen if JWTMiddleware ran first, but be safe
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
b := l.getBucket(userID)
if !b.allow() {
w.Header().Set("Retry-After", "1")
http.Error(w, `{"error":"rate limit exceeded, please slow down"}`, http.StatusTooManyRequests)
return
}
next.ServeHTTP(w, r)
})
}
// Must match the key type in auth/middleware.go
type contextKey string
```
**Step 2 — Wire it into `routes/routes.go`:**
```go
// In the route setup function, after creating JWTMiddleware:
limiter := ratelimit.NewLimiter(10, 2) // 10 burst, 2 req/sec sustained
// Example: wrap the recommendations handler
mux.Handle("/recommendations",
auth.JWTMiddleware(jwtSecret,
limiter.Middleware(
http.HandlerFunc(recHandler.GetRecommendation),
),
),
)
```
The middleware chain is: `JWTMiddleware → RateLimitMiddleware → Handler`. This order matters — rate limiting only makes sense for authenticated users (known by user ID).
**Step 3 (Bonus) — Add periodic cleanup to avoid unbounded memory growth:**
If the service runs for weeks, the `buckets` map grows forever (one entry per unique user). Add a cleanup goroutine that removes buckets for users who haven't made a request in a configurable duration (e.g., 1 hour).
---
### 🏁 Acceptance Criteria
- [ ] A `Limiter` struct exists in `ratelimit/limiter.go` using the token bucket algorithm
- [ ] The `bucket.allow()` method is correct: refills on elapsed time, caps at capacity, returns `false` when empty
- [ ] Per-user state is stored safely — bucket creation and token consumption are both protected by mutexes
- [ ] The middleware returns **HTTP 429** with a `Retry-After: 1` header when the limit is exceeded
- [ ] The middleware reads `user_id` from the request context (set by `JWTMiddleware`)
- [ ] At least the `/recommendations` route is wrapped with the rate limiter
- [ ] `go test -race ./internal/ratelimit/...` passes with zero data races
---
### 💡 Technical Hints
- Look at `backend/core_service/internal/auth/middleware.go` for how `user_id` is stored in context using `context.WithValue`. Your middleware reads that same key. **Be careful**: you must use the same `contextKey` type, otherwise `r.Context().Value("user_id")` will return `nil` because Go context keys are compared by both value AND type.
- The **double-checked locking** pattern in `getBucket` (check RLock → check WLock) is important for concurrent correctness and performance. Without it, acquiring a write lock on every request would be a bottleneck.
- Test your rate limiter by writing a `TestRateLimiter` in `limiter_test.go` that fires 15 requests rapidly and asserts that the first 10 succeed and the next 5 are rejected.
---
### 🚀 Getting Started
1. Fork the repository
2. Create a branch: `git checkout -b feat/issue-29-rate-limiting`
3. Create `backend/core_service/internal/ratelimit/limiter.go`
4. Write tests in `backend/core_service/internal/ratelimit/limiter_test.go`
5. Wire into `routes/routes.go`
6. Run: `cd backend/core_service && go test -race ./...`
7. Open a PR with test output showing the rate limiter working!
Contributor guide
Research direction
Start with backend/core_service/internal/auth/middleware.go to verify how user_id is stored in request context, then create the limiter in backend/core_service/internal/ratelimit/limiter.go and wire it in backend/core_service/routes/routes.go. Add limiter_test.go for concurrent and burst behavior, run cd backend/core_service && go test -race ./..., and consider the acceptance criteria met when authenticated recommendations requests are limited per user and excess requests return HTTP 429 with Retry-After: 1.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100