OWASP / OWASP/Nest

feat: Add structured logging and Sentry tracing to AI agent workflows

Open
#4,888 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
451
Forks
707
Avg merge
22h 59m
Merged PRs (30d)
91

Description

**Is your feature request related to a problem? Please describe.**

The AI agent RAG workflow in `apps/ai/agent/agent.py` and `apps/ai/agent/nodes.py` currently has no observability instrumentation. There is no visibility into per-node latency, OpenAI API call duration, model error rates, or iteration counts. This makes it impossible to debug slow AI responses, identify failing model calls, or track AI performance metrics in production or staging.

**Describe the solution you'd like**

Add structured logging with `extra` context and Sentry performance tracing spans to the AI agent's RAG workflow:

- In `agent.py`: Wrap the full `graph.invoke()` in a Sentry span (`op="ai.rag_workflow"`) with tags for query, iterations, and duration. Add structured log entries at workflow start and completion with duration and iteration count.

- In `nodes.py`:

- `retrieve()` — Sentry span with tags for query, retrieval limit, chunks retrieved/filtered, and duration.

- `generate()` — Sentry span with tags for iteration, feedback flag, context chunk count, generation duration, answer length.

- `evaluate()` — Sentry span with tags for iteration, completeness, re-retrieval flag.

- `extract_query_metadata()` — Sentry span around the OpenAI call with model name, duration, error tracking, and structured logging on success/failure.

- `call_evaluator()` — Sentry span around the OpenAI call with model name, duration, error tracking, and structured logging on success/failure.

All log calls use `logger.info(...)` / `logger.error(...)` with `extra` dicts for structured fields, compatible with the existing `JsonLogFormatter` in `base.py`.

**Describe alternatives you've considered**

- Adding OpenTelemetry spans instead of Sentry spans — considered but Sentry is already configured (`sentry_sdk` in dependencies, initialized in staging.py/production.py), so using Sentry spans is consistent with the existing stack and avoids adding new dependencies.

- Plain logging without Sentry — considered but provides no trace waterfall visualization, making performance debugging harder.

**Are you going to work on implementing this?**

- [x] Yes

- [ ] No

**Additional context**

This is part of the GSoC 2026 observability project for OWASP Nest. The Django request-logging middleware (`RequestLoggingMiddleware`, `JsonLogFormatter`, `RequestLoggingFilter`) was already implemented in a prior PR. This extension brings the same observability standard to the AI agent code paths.

Branch: https://github.com/Divyateja2709/Nest/tree/new-update

```

Contributor guide

Open the contributing guide

Research direction

Start by reading apps/ai/agent/agent.py and apps/ai/agent/nodes.py, then inspect the existing JsonLogFormatter in base.py and Sentry initialization in staging.py and production.py. Implement the requested spans and structured logs around the named workflow and OpenAI entry points, with the listed fields and error tracking. Done means the RAG workflow exposes timing, iteration, retrieval, generation, evaluation, and model-call observability without adding dependencies.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.