aws / aws/amazon-q-developer-cli

Scripting improvements

Open
#2,809 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2k
Forks
439
PR merge metrics
No merged PRs in 30d

Description

This week I tried to use `q chat` in a script - I had a batch of problems with clear testing criteria, so I called q chat for each problem, and ran the tests. I ran into a number of issues

The first issue is that q chat won't exit when a task is done, it always returns to a prompt. It will give hints to exit with /quit or Ctrl+C but it won't exit. It will exit if you use `--no-interactive` but that brings us to the second issue.

q chat --no-interactive will frequently fail after trying to use a tool that requires approval. Often there are other ways to achieve the goal using approved tools (or arguments), but q cli just errors. The most common examples I have are multi-line commands, which do not match `.*` in allowedCommands.

```sh
git commit -m "some message that spans multiple lines

because it has a subject and body"
```

* Have an interactive mode where the LLM can exit when complete, but I can approve tool use instead of crashing the program.
* Allow the llm to retry if a tool-call would require approval in --no-interactive
* Improve introspection so the LLM is more likely to use tool arguments that are allowed
* Improve the execute_bash allowedCommand regexes (or at-least the docs)

---
Some more out-there ideas to improve q chat's use in scripts would be a way to support stdio redirection and setting the exit code. q chat would have to send all output to stderr, while sending the 'results' to stdout. The results could even support JSONSchema.

* Flag to send all output to stderr
* built-in tool to exit q chat with a specific exit code
* built-in tool that writes arguments to stdout
* The tool's argument schema would be defined by the CLI args. JSONSchema might be too verbose for a CLI so maybe a small DSL?

```
#!/bin/bash
set -euo pipefail

output=$(q chat --output=stderr --result-schema="{result: number}" "return the sum of two and two")
# q chat's thought process would be printed to stderr
# stderr > > python
# stderr > return 2+2
#
# the final result call would be printed to stdout, and captured by output.

echo $output
# stdout > 4
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.