Feat: Port the agent optimization module so agents can automatically improve their own prompts (parity with adk-python optimization)
- Dominant language
- TypeScript
- Stars
- 1.4k
- Forks
- 205
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 92
Description
## Summary
adk-python can automatically rewrite an agent's system prompt to make it score better. adk-js cannot, because the `optimization/` module was never ported.
This issue proposes porting the core of that module plus `SimplePromptOptimizer`. It is about 450 lines of Python, needs no external library, and does not require the eval framework.
**Is your feature request related to a problem? Please describe.**
Tuning an agent's `instruction` today is guesswork. You edit the text, run the agent a few times, and judge by feel. There is no way to measure whether a change helped, and no way to search for a better prompt automatically.
adk-python solves this with `src/google/adk/optimization/`. adk-js has no equivalent. The directory does not exist.
| adk-python | adk-js |
| --- | --- |
| `optimization/agent_optimizer.py` | missing |
| `optimization/sampler.py` | missing |
| `optimization/data_types.py` | missing |
| `optimization/simple_prompt_optimizer.py` | missing |
| `optimization/local_eval_sampler.py` | missing |
| `optimization/gepa_root_agent_optimizer.py` | missing |
| `optimization/gepa_root_agent_prompt_optimizer.py` | missing |
**Describe the solution you'd like**
Add `core/src/optimization/` with the optimizer core and one working optimizer.
### What it does
`SimplePromptOptimizer` is hill climbing over prompt text:
1. Score the agent as it is now to get a baseline.
2. Ask an LLM to rewrite the instruction, telling it the current score.
3. Build a candidate agent using the new instruction.
4. Score the candidate on a random batch of training examples.
5. Keep the candidate only if it beats the current best, otherwise discard it.
6. Repeat for `numIterations` rounds.
7. Score the winner against a held out validation set and return it.
Concretely, an instruction like this:
```
You are a support agent. Help the user.
```
gets rewritten into something like this, because that version measurably scores higher:
```
You are a customer support agent.
Always ask for an order number before taking any action.
Check the refund policy before issuing a refund.
Orders older than 30 days are not eligible.
```
### Proposed API
```ts
const optimizer = new SimplePromptOptimizer({
optimizerModel: 'gemini-2.5-flash',
numIterations: 10,
batchSize: 5,
});
const result = await optimizer.optimize(myAgent, mySampler);
result.optimizedAgents[0].optimizedAgent; // agent with the improved instruction
result.optimizedAgents[0].overallScore; // score on the validation set
```
### Scoring stays pluggable
The framework never decides what "good" means. The developer implements a `Sampler`:
```ts
export abstract class Sampler {
abstract getTrainExampleIds(): string[];
abstract getValidationExampleIds(): string[];
abstract sampleAndScore(
candidate: LlmAgent,
exampleSet?: 'train' | 'validation',
batch?: string[],
captureFullEvalData?: boolean,
): Promise;
}
```
This matters for scoping, so I want to be explicit about it. In adk-python, `Sampler` is an abstract base class and `LocalEvalSampler` is just one implementation of it that happens to use the eval framework. Checking the imports confirms the coupling is almost nonexistent everywhere else:
| File | Lines | Imports from `evaluation/` |
| --- | --- | --- |
| `data_types.py` | 90 | none |
| `sampler.py` | 73 | none |
| `agent_optimizer.py` | 49 | none |
| `simple_prompt_optimizer.py` | 231 | one retry helper |
| `gepa_root_agent_prompt_optimizer.py` | 325 | one error message constant |
| `local_eval_sampler.py` | 372 | heavy, 16 imports |
So the eval dependency lives in exactly one optional file. **adk-js can have working optimization without the eval framework**, with users supplying their own scoring function. If an eval framework lands later, a `LocalEvalSampler` slots in behind the same interface without touching any optimizer.
### Scope
**In scope**
- `data_types.ts`: `SamplingResult`, `UnstructuredSamplingResult`, `AgentWithScores`, `OptimizerResult`
- `sampler.ts`: the `Sampler` abstract class
- `agent_optimizer.ts`: the `AgentOptimizer` abstract class
- `simple_prompt_optimizer.ts`: the loop above
- Unit tests with a stub LLM, and one runnable sample
**Out of scope, deliberately**
- **GEPA optimizers.** adk-python depends on the external `gepa>=0.1` pip package and adapts to it. There is no official JS build. Several third party TypeScript ports exist on npm (`@currentai/dsts`, `@astragenie/gepa-core`, `@nightowlsdev/gepa`, `gepa-ts`) but they vary a lot in maturity, and `gepa-rpc` takes a different approach by bridging to the Python implementation. That dependency choice deserves its own issue rather than being buried in this one.
- **`LocalEvalSampler`**, since it needs the eval framework.
- **The eval framework itself.**
### One blocker that needs a decision
Python's optimizer creates each candidate like this:
```python
candidate_agent = best_agent.clone(update={"instruction": new_prompt_text})
```
**adk-js has no `clone()` on any agent class.** There are zero matches for `clone(` across `core/src/agents/`.
It cannot be ported directly either. adk-python's version is built on pydantic (`model_fields`, `model_copy`), while adk-js uses plain classes with `readonly` fields. On top of that, the adk-js `LlmAgent` constructor derives state rather than just assigning it: it normalizes callbacks with `getCannonicalCallback()`, converts schemas with `isZodObject()`, builds and then mutates `requestProcessors` with `splice` and `push`, conditionally appends `AGENT_TRANSFER_LLM_REQUEST_PROCESSOR`, and rewires children via `setParentAgentForSubAgents()`.
Because the instance is a different shape from the config that created it, two obvious shortcuts do not work. Shallow copying the instance would duplicate already mutated processor arrays and share references with the original. Rebuilding a config from the instance fields is lossy, since the original config cannot be recovered from derived state.
Three options:
| Option | Approach | Trade-off |
| --- | --- | --- |
| **A** | Store the config on `BaseAgent` and clone by re-running the constructor with overrides | Correct, and matches adk-python's API exactly. Touches core. |
| **B** | Skip `clone()`. The optimizer takes an agent factory instead | No core changes, but diverges from adk-python's `optimize(initial_agent, sampler)` signature |
| **C** | Add a narrow `LlmAgent.withInstruction()` helper | Smallest change, but invents an API that adk-python does not have |
I lean toward **A**, shipped as a separate PR first. `clone()` is a parity gap in its own right (adk-python has it on `BaseAgent`), and it has value beyond optimization for agent templating, per-tenant agent variants, and test fixtures. But this is a core class, so I would rather agree on the approach here than send a surprise PR.
**Describe alternatives you've considered**
**Doing nothing and letting users hand tune prompts.** This is the status quo. It works, but it is unmeasured and does not scale past a few prompts. The whole point of the module is replacing intuition with a score.
**Waiting for the eval framework to land first.** This was my initial assumption, and the import graph above disproved it. Optimization is usable now with a user supplied `Sampler`, and gating it on a much larger port would delay it for no technical reason.
**Starting with GEPA instead**, since it is the more capable optimizer. I decided against it because the dependency question is genuinely unsettled in JavaScript and would stall the whole thing. `SimplePromptOptimizer` needs nothing external, so it can land first and prove the interfaces. GEPA then slots into the same `AgentOptimizer` base.
**Additional context**
These modules are marked `@experimental` in adk-python, and I would suggest carrying that marker over so expectations are clear.
A few implementation details worth recording so they are not lost:
- The response reader must skip thought parts (`if part.text and not part.thought`), otherwise reasoning text leaks into the generated prompt.
- `batchSize` larger than the number of training examples is clamped, with a warning.
- Empty scores return `0.0` rather than dividing by zero.
- Defaults to preserve: `optimizerModel: 'gemini-2.5-flash'`, `numIterations: 10`, `batchSize: 5`, and thinking config with `includeThoughts: true` and `thinkingBudget: 10240`.
- `simple_prompt_optimizer.py`'s only eval import is `add_default_retry_options_if_not_present`. It can be inlined rather than pulling in an eval namespace.
**This is not free to run and the docs should say so.** Ten iterations at a batch of five means fifty or more agent runs plus ten rewriter calls. That is normal for this category of tool, but users should not be surprised by the bill.
**Happy to implement this and open a PR** (CLA already signed). Three questions before I start:
1. `clone()` on `BaseAgent` (Option A), or an agent factory in the optimizer (Option B)?
2. Should this land in `core/src/optimization/`, or somewhere else given it is experimental?
3. Is `SimplePromptOptimizer` first with GEPA deferred to a separate issue the sequencing you want?
Contributor guide
Assessment
This issue has not been assessed yet.