getsentry / getsentry/sentry-mcp

Seer AI analysis suggestions may not always be accurate

Open
#415 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
853
Forks
144
Avg merge
19h 25m
Merged PRs (30d)
32

Description

## Summary

When analyzing errors in the search_events agent, Seer AI suggested adding `maxToolRoundtrips: 3` to the `generateText` call. However, this property doesn't exist in the AI SDK's generateText options, leading to TypeScript errors.

## Problem

1. Seer AI provides analysis and suggestions that may not always be accurate or applicable
2. Our tools should not blindly trust Seer's suggestions without validation
3. We need better error messaging to indicate when Seer's suggestions might be incorrect

## Context

From issue MCP-SERVER-ECD: https://sentry.sentry.io/issues/6761713004/
- Error: `AI_NoObjectGeneratedError: No object generated: could not parse the response.`
- Seer suggested: "Add maxToolRoundtrips: 3 to generateText call"
- Reality: This property doesn't exist in the AI SDK

## Proposed Solution

1. Add disclaimers in the analyze_issue_with_seer tool output indicating that suggestions should be validated
2. Include context about the tool's limitations in the response
3. Consider adding validation for common incorrect suggestions

## Related Code

- packages/mcp-server/src/tools/search-events/agent.ts:491-499
- packages/mcp-server/src/tools/analyze-issue-with-seer/handler.ts

## Labels

- bug
- tool-improvement
- seer-ai

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.