Monitoring: Automate alerts and dashboards

Context

Establishing the ongoing monitoring/observability structure for Kaapi. Akhilesh's health-check doc is taken as the infra baseline checklist. Direction is to automate rather than manually check.

Scope

• Push breach notifications (memory, CloudWatch alarms) to Discord instead of manual checks.
• Eval-run load monitoring:
  stage 1 = Sentry HTTP endpoint graphs on eval-run endpoints as a proxy for run density;
  stage 2 = live Celery queue backlog alerts. No LLM-call rate limits yet — deliberately deferred until scale demands it.
• App-level monitoring: endpoint hit counts, slow requests, waterfall dig-ins; build a custom Sentry dashboard.
• Target: two dashboards — one AWS (infra), one Sentry (app-level) — reviewed in the weekly call.
• Weekly reporting convention: share only red flags / odd numbers; skip steadily-green metrics.

Actions

• Team to propose feature-wise / app-level metrics (latency etc.) for the Sentry dashboard.
• Set up Discord breach notifications.
• Verify staging Container Insights and configure staging alerts.

Related: ProjectTech4DevAI/kaapi-backend#1052, ProjectTech4DevAI/kaapi-backend#1005, ProjectTech4DevAI/kaapi-backend#825