google-gemini / google-gemini/gemini-cli

Agent should run changes to validate app

Open
#17,110 5 comments 0 reactions 0 assignees View on GitHub
🔒 maintainer only area/agent kind/enhancement priority/p2 status/bot-triaged workstream-rollup
Dominant language
TypeScript
Stars
107k
Forks
14.6k
Avg merge
2d 3h
Merged PRs (30d)
45

Description

LLMs are by nature non-deterministic and imperfect. My experience has been that the key to making an effective AI coding agent is to provide it with effective feedback loops. i.e.: the ability self-validate its changes. This feature tracks steering the agent to self-validate its changes.

**Scope of work:**
- Behavioral evals test that the agent "runs" any code changes it made. Ideally the actual app, and not just the tests.
- Prompt changes, as needed, to steer it in that direction.

**Examples:**
- If changes are to a console app, the agent should run the console app and validate its change.
- If changes are to a REST API, the agent should run the API in the background and exercise it with CURL and check for proper behavior.
- Potential overlap with: https://github.com/google-gemini/gemini-cli/issues/15956

**North star**

I really want to see Gemini CLI able to run itself to validate its own changes to itself.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.