google-gemini / google-gemini/gemini-cli
Agent should run changes to validate app
- Dominant language
- TypeScript
- Stars
- 107k
- Forks
- 14.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 45
Description
LLMs are by nature non-deterministic and imperfect. My experience has been that the key to making an effective AI coding agent is to provide it with effective feedback loops. i.e.: the ability self-validate its changes. This feature tracks steering the agent to self-validate its changes.
**Scope of work:**
- Behavioral evals test that the agent "runs" any code changes it made. Ideally the actual app, and not just the tests.
- Prompt changes, as needed, to steer it in that direction.
**Examples:**
- If changes are to a console app, the agent should run the console app and validate its change.
- If changes are to a REST API, the agent should run the API in the background and exercise it with CURL and check for proper behavior.
- Potential overlap with: https://github.com/google-gemini/gemini-cli/issues/15956
**North star**
I really want to see Gemini CLI able to run itself to validate its own changes to itself.
Contributor guide
Assessment
This issue has not been assessed yet.