AgentOps-AI / AgentOps-AI/agentops

[Feature]: GTM / product team dashboard — non-engineer view of agent session health

Abierto
#1,383 5 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
5.8k
Forks
619
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Problem

AgentOps is currently an engineering-first tool. The dashboard surfaces rich technical data — session traces, LLM costs, step waterfalls — but it's not usable by the people who often need to understand agent health the most: product managers, customer success teams, and GTM leads.

In practice, when an AI product ships to customers, the people fielding feedback ("the agent got stuck", "it completed the wrong task", "it's too slow") are not the engineers who built it. Today there's no way for a non-engineer to open AgentOps and quickly answer:
- Is the agent working for most users right now?
- Which user sessions are failing and why?
- Has quality degraded since last week's model update?
- What's the error rate today vs. yesterday?

## Proposed Solution

A **simplified "Product Health" view** in the AgentOps dashboard, separate from the full trace explorer, that surfaces:

1. **Session success rate** — % of sessions that completed vs. failed/timed out, with a 7-day trend line
2. **Top failure reasons** — grouped by error type (timeout, loop detected, action failed, LLM error) in plain English
3. **User-facing impact** — if `user_id` metadata is set, show how many distinct users were affected by failures
4. **Quality score trend** — if eval metrics are configured, show average score over time as a simple sparkline
5. **Shareable snapshot** — a link to share a read-only summary (for Slack updates, stakeholder reviews, incident reports)

This doesn't require new data collection — AgentOps already captures everything needed. It's a presentation layer built on top of existing sessions and metadata.

## Why This Matters for GTM

As teams sell AI agent products to enterprise buyers, trust and reliability reporting becomes a procurement requirement. A PM-friendly view of agent health would help AgentOps expand from "dev tool" to "AI operations platform" — a significantly larger positioning with broader buying motion.

Happy to help think through the UX or write a product spec for this.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.