feat: Add structured logging and Sentry tracing to AI agent workflows
- Dominant language
- Python
- Stars
- 451
- Forks
- 707
- Avg merge
- 22h 59m
- Merged PRs (30d)
- 91
Description
**Is your feature request related to a problem? Please describe.**
The AI agent RAG workflow in `apps/ai/agent/agent.py` and `apps/ai/agent/nodes.py` currently has no observability instrumentation. There is no visibility into per-node latency, OpenAI API call duration, model error rates, or iteration counts. This makes it impossible to debug slow AI responses, identify failing model calls, or track AI performance metrics in production or staging.
**Describe the solution you'd like**
Add structured logging with `extra` context and Sentry performance tracing spans to the AI agent's RAG workflow:
- In `agent.py`: Wrap the full `graph.invoke()` in a Sentry span (`op="ai.rag_workflow"`) with tags for query, iterations, and duration. Add structured log entries at workflow start and completion with duration and iteration count.
- In `nodes.py`:
- `retrieve()` — Sentry span with tags for query, retrieval limit, chunks retrieved/filtered, and duration.
- `generate()` — Sentry span with tags for iteration, feedback flag, context chunk count, generation duration, answer length.
- `evaluate()` — Sentry span with tags for iteration, completeness, re-retrieval flag.
- `extract_query_metadata()` — Sentry span around the OpenAI call with model name, duration, error tracking, and structured logging on success/failure.
- `call_evaluator()` — Sentry span around the OpenAI call with model name, duration, error tracking, and structured logging on success/failure.
All log calls use `logger.info(...)` / `logger.error(...)` with `extra` dicts for structured fields, compatible with the existing `JsonLogFormatter` in `base.py`.
**Describe alternatives you've considered**
- Adding OpenTelemetry spans instead of Sentry spans — considered but Sentry is already configured (`sentry_sdk` in dependencies, initialized in staging.py/production.py), so using Sentry spans is consistent with the existing stack and avoids adding new dependencies.
- Plain logging without Sentry — considered but provides no trace waterfall visualization, making performance debugging harder.
**Are you going to work on implementing this?**
- [x] Yes
- [ ] No
**Additional context**
This is part of the GSoC 2026 observability project for OWASP Nest. The Django request-logging middleware (`RequestLoggingMiddleware`, `JsonLogFormatter`, `RequestLoggingFilter`) was already implemented in a prior PR. This extension brings the same observability standard to the AI agent code paths.
Branch: https://github.com/Divyateja2709/Nest/tree/new-update
```
Contributor guide
Research direction
Start by reading apps/ai/agent/agent.py and apps/ai/agent/nodes.py, then inspect the existing JsonLogFormatter in base.py and Sentry initialization in staging.py and production.py. Implement the requested spans and structured logs around the named workflow and OpenAI entry points, with the listed fields and error tracking. Done means the RAG workflow exposes timing, iteration, retrieval, generation, evaluation, and model-call observability without adding dependencies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100