google / google/adk-python

[QUESTION] AgentEvaluator.evaluate_eval_set

未关闭
#1,792 2 条评论 0 个 reaction 已指派 1 人 已被 @ankursharmas 认领 在 GitHub 查看
eval needs review
主要语言
Python
星标
21.5k
派生
4k
平均合并
1 天 14 小时
30 天内合并 PR
37

描述

# Description:

We're building an HTTP endpoint that uses AgentEvaluator from ADK to evaluate an agent on-demand. The challenge we're facing is that the evaluator currently requires an agent_module (i.e., a Python module path), and not a preconstructed agent instance.

In our case, we want to initialize the agent with runtime-specific parameters — for example, an account_id extracted from an HTTP header. This value is dynamic and cannot be hardcoded in the agent module, nor is it available at module load time.

However, since AgentEvaluator.evaluate(...) only accepts the module path, we're unable to pass this context into the agent. The agent has to be constructed as the "root agent" in a fixed folder structure.

What we need / feature request:

It would be very helpful if AgentEvaluator supported an additional option to provide an already-initialized agent instance, like:

```python
agent = MyAgent(account_id="abc-123")
AgentEvaluator.evaluate_eval_set(agent=agent, ...)
```
### Workarounds we've tried:

Dynamic agent module generation: We’ve also tried programmatically generating the agent directory structure (that constructs the agent with injected values), then importing and using that as the agent_module. This technically works, but it's weird.

# Question

Is there currently any recommended way to achieve this kind of dynamic initialization for evaluation use cases like this?
If there's an alternative pattern that would let us pass info (like account_id) into the agent at runtime while still using AgentEvaluator, we’d love to hear it.

Thanks a lot in advance! 🙏

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。