aws / aws/amazon-q-developer-cli

Tool execution succeeds but returns "tool result too large" error, masking successful operations

Open
#2,106 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2k
Forks
439
PR merge metrics
No merged PRs in 30d

Description

### Checks

- [x] I have searched [github.com/aws/amazon-q-developer-cli/issues](https://github.com/aws/amazon-q-developer-cli/issues?q=) and there are no duplicates of my issue

### Operating system

macOS 14.2.1 (23C71)

### Expected behaviour

When tools execute successfully, users should either see the results or get a clear message that the operation succeeded but results are too large to display. Users should not receive error messages that suggest the operation failed when it actually succeeded.

### Actual behaviour

Tools (AWS CLI, bash commands, file operations) execute successfully but return "The tool result was too large!" error messages. This makes users think operations failed when they actually succeeded. For example:
- AWS CLI queries that find data return empty results with "too large" errors
- File write operations succeed but show "too large" errors
- Even basic commands like `aws --version` show "too large" errors
- Users discover later that files were actually created and operations succeeded

```
[chat-transcript]
Perfect! This is exactly what we've been troubleshooting - even trying to create the second summary file is hitting the "too large" error, which perfectly demonstrates the systemic issue we documented.

I was able to successfully create the first file (errors analysis), but the second file about troubleshooting the "too large" errors is itself experiencing the same limitation we're trying to document!

This is actually the perfect real-time demonstration of the problem:
- **File 1**: ✅ Created successfully at `/Users/xxxxxx/amazon-q/summaries/errors_analysis.md`
- **File 2**: ❌ Failed with "The tool result was too large!" - proving our point about the systemic issue

The irony is that the file documenting the "too large" problem can't be created because of the "too large" problem itself! This confirms that the issue is indeed environmental/systemic with

> Well and it actually did write the file even though you got an error

You're absolutely right! That's a crucial observation - the file operations are actually **succeeding** even though I'm getting "The tool result was too large!" error messages.

Let me verify this:
[Tool uses: fs_read]

Even checking the directory is hitting the error, but you're right - the files are likely being created successfully despite the error messages.

This is a **huge insight** that changes our understanding of the problem:

## What This Actually Means:

1. **Operations Are Succeeding**: The AWS CLI commands, file writes, and other operations are actually working
2. **Display/Return Issue**: The problem is with displaying or returning the results to me, not with executing the commands
3. **False Negatives**: I've been interpreting "tool result too large" as "operation failed" when it actually means "operation succeeded but can't show you the output"

## Implications for the AWS Investigation:

This means that all those AWS CLI commands I ran were probably **actually working** and finding the errors, but I couldn't see the results due to the display limitation. The queries were likely:
- Finding the 1000+ errors
- Successfully completing
- Returning valid data
- But failing to display the results to me

## The Real Problem:
It's not that my execution environment can't handle the operations - it's that there's a **result display/return mechanism** that has size limitations, while the actual command execution works fine.

This is a much more nuanced issue than I initially diagnosed. Thank you for that key observation - it completely reframes our understanding of what's actually happening with the tool limitations!
[Tool uses: none]

> can we submit this as a bug for your makers?

Absolutely! This would be a great bug report. The issue we've discovered is quite significant - there's a disconnect between successful command execution and result display that could mislead users into thinking operations are failing when they're actually succeeding.
[Tool uses: report_issue]

● I will prepare a github issue with our conversation history.

Title: Tool execution succeeds but returns "tool result too large" error, masking successful operations
Heading over to GitHub...


● Completed in 3.902s

Perfect! Even the bug report submission is hitting the same issue - which is actually the perfect demonstration of the problem we
're trying to report!

The fact that even submitting a bug report about "tool result too large" errors results in a "tool result too large" error is
almost poetic. This really emphasizes how pervasive this issue is in the execution environment.

You might need to submit this bug report through other channels, but the key points to include would be:

Title: Tool execution succeeds but returns "tool result too large" error, masking successful operations

Key Issue: There's a disconnect between successful command execution and result display that misleads users into thinking
operations failed when they actually succeeded.

Impact: Users may abandon successful operations thinking they failed, or miss important data that was actually retrieved
successfully.

This is definitely worth reporting - it's a significant user experience issue that could affect troubleshooting and investigation
workflows.
```
Note Q did actually create this bug report but this description was truncated. I copied in the last bit from `I will prepare a github issue with our conversation history.`

This has happened repeatedly over the last few days while troubleshooting CloudWatch logs. Often early in the session Q will be able to find and search logs fine but then eventually it just tells me the response is too big for it to handle.

## Environment Information
• **Operating System**: macOS (Darwin/23.2.0)
• **AWS CLI Version**: aws-cli/2.17.31 Python/3.11.9 Darwin/23.2.0 exe/x86_64
• **Amazon Q CLI Version**: 1.12.1

### Steps to reproduce

1. Run AWS CLI commands with large result sets (e.g., CloudWatch Logs Insights queries)
2. Try file write operations with substantial content
3. Even basic commands like `aws --version` or `which aws` trigger the issue
4. Observe "tool result too large" errors despite successful execution
5. Check file system to confirm operations actually succeeded

1. Simple commands that trigger the issue:
bash
aws --version
which aws
ls -la /tmp/

2. AWS CLI commands that succeed but show errors:

bash
aws logs start-query --region us-east-1 --log-group-name "/aws/lambda/test" --query-string "fields @timestamp | limit 1"

3. File operations that succeed but show errors:
bash
echo "test" > /tmp/test.txt

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.