AgentOps-AI / AgentOps-AI/agentops

[Feature]: GTM / product team dashboard — non-engineer view of agent session health

未关闭
#1,383 5 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
5.8k
派生
619
PR 合并指标
30 天内没有已合并 PR

描述

## Problem

AgentOps is currently an engineering-first tool. The dashboard surfaces rich technical data — session traces, LLM costs, step waterfalls — but it's not usable by the people who often need to understand agent health the most: product managers, customer success teams, and GTM leads.

In practice, when an AI product ships to customers, the people fielding feedback ("the agent got stuck", "it completed the wrong task", "it's too slow") are not the engineers who built it. Today there's no way for a non-engineer to open AgentOps and quickly answer:
- Is the agent working for most users right now?
- Which user sessions are failing and why?
- Has quality degraded since last week's model update?
- What's the error rate today vs. yesterday?

## Proposed Solution

A **simplified "Product Health" view** in the AgentOps dashboard, separate from the full trace explorer, that surfaces:

1. **Session success rate** — % of sessions that completed vs. failed/timed out, with a 7-day trend line
2. **Top failure reasons** — grouped by error type (timeout, loop detected, action failed, LLM error) in plain English
3. **User-facing impact** — if `user_id` metadata is set, show how many distinct users were affected by failures
4. **Quality score trend** — if eval metrics are configured, show average score over time as a simple sparkline
5. **Shareable snapshot** — a link to share a read-only summary (for Slack updates, stakeholder reviews, incident reports)

This doesn't require new data collection — AgentOps already captures everything needed. It's a presentation layer built on top of existing sessions and metadata.

## Why This Matters for GTM

As teams sell AI agent products to enterprise buyers, trust and reliability reporting becomes a procurement requirement. A PM-friendly view of agent health would help AgentOps expand from "dev tool" to "AI operations platform" — a significantly larger positioning with broader buying motion.

Happy to help think through the UX or write a product spec for this.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。