google / google/adk-python

[QUESTION] AgentEvaluator.evaluate_eval_set

Aperta
#1,792 2 commenti 0 reazioni 1 assegnatario Rivendicata da @ankursharmas Vedi su GitHub
eval needs review
Lingua principale
Python
Stelle
21.5k
Fork
4k
Merge medio
1g 14h
PR unite (30g)
37

Descrizione

# Description:

We're building an HTTP endpoint that uses AgentEvaluator from ADK to evaluate an agent on-demand. The challenge we're facing is that the evaluator currently requires an agent_module (i.e., a Python module path), and not a preconstructed agent instance.

In our case, we want to initialize the agent with runtime-specific parameters — for example, an account_id extracted from an HTTP header. This value is dynamic and cannot be hardcoded in the agent module, nor is it available at module load time.

However, since AgentEvaluator.evaluate(...) only accepts the module path, we're unable to pass this context into the agent. The agent has to be constructed as the "root agent" in a fixed folder structure.

What we need / feature request:

It would be very helpful if AgentEvaluator supported an additional option to provide an already-initialized agent instance, like:

```python
agent = MyAgent(account_id="abc-123")
AgentEvaluator.evaluate_eval_set(agent=agent, ...)
```
### Workarounds we've tried:

Dynamic agent module generation: We’ve also tried programmatically generating the agent directory structure (that constructs the agent with injected values), then importing and using that as the agent_module. This technically works, but it's weird.

# Question

Is there currently any recommended way to achieve this kind of dynamic initialization for evaluation use cases like this?
If there's an alternative pattern that would let us pass info (like account_id) into the agent at runtime while still using AgentEvaluator, we’d love to hear it.

Thanks a lot in advance! 🙏

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.