google / google/adk-python

[QUESTION] AgentEvaluator.evaluate_eval_set

オープン
#1,792 コメント 2 件 リアクション 0 件 担当者 1 名 @ankursharmas が担当を希望しています GitHub で見る
eval needs review
主要言語
Python
スター
21.5k
フォーク
4k
平均マージ
1日 14時間
マージ済み PR(30日)
37

説明

# Description:

We're building an HTTP endpoint that uses AgentEvaluator from ADK to evaluate an agent on-demand. The challenge we're facing is that the evaluator currently requires an agent_module (i.e., a Python module path), and not a preconstructed agent instance.

In our case, we want to initialize the agent with runtime-specific parameters — for example, an account_id extracted from an HTTP header. This value is dynamic and cannot be hardcoded in the agent module, nor is it available at module load time.

However, since AgentEvaluator.evaluate(...) only accepts the module path, we're unable to pass this context into the agent. The agent has to be constructed as the "root agent" in a fixed folder structure.

What we need / feature request:

It would be very helpful if AgentEvaluator supported an additional option to provide an already-initialized agent instance, like:

```python
agent = MyAgent(account_id="abc-123")
AgentEvaluator.evaluate_eval_set(agent=agent, ...)
```
### Workarounds we've tried:

Dynamic agent module generation: We’ve also tried programmatically generating the agent directory structure (that constructs the agent with injected values), then importing and using that as the agent_module. This technically works, but it's weird.

# Question

Is there currently any recommended way to achieve this kind of dynamic initialization for evaluation use cases like this?
If there's an alternative pattern that would let us pass info (like account_id) into the agent at runtime while still using AgentEvaluator, we’d love to hear it.

Thanks a lot in advance! 🙏

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。