google / google/adk-docs

Add MLflow evaluation scorer documentation for ADK evaluators

Open
#2,117 1 comment 0 reactions 2 assignees Claimed by @ankursharmas View on GitHub
Dominant language
Shell
Stars
1.5k
Forks
1.3k
Avg merge
7d 1h
Merged PRs (30d)
34

Description

## 🔴 Required Information

### Is your feature request related to a specific problem?

ADK's evaluation criteria (ToolTrajectory, ResponseMatch, Hallucinations, etc.) are powerful but users who also use MLflow for experiment tracking have no way to run ADK evaluators through `mlflow.genai.evaluate()`. MLflow already has ADK tracing integration but no evaluation scorer integration.

### Describe the Solution You'd Like

Add a documentation page to ADK docs showing how to use ADK evaluators as MLflow scorers. An MLflow integration has been submitted as PR [mlflow/mlflow#22299](https://github.com/mlflow/mlflow/pull/22299) that wraps ADK's `TrajectoryEvaluator` and `RougeEvaluator` as MLflow third-party scorers.

Example usage:

```python
from mlflow.genai.scorers.google_adk import ToolTrajectory, ResponseMatch

results = mlflow.genai.evaluate(
data=eval_dataset,
scorers=[
ToolTrajectory(match_type="EXACT", threshold=0.5),
ResponseMatch(threshold=0.6),
],
)
```

A docs page under `docs/evaluate/` or `docs/integrations/` showing this integration would help ADK users who track experiments with MLflow.

### Impact on your work

This enables ADK users to evaluate agents through MLflow's unified evaluation pipeline, combining ADK's deterministic evaluators with MLflow's experiment tracking, tracing, and comparison tools.

### Willingness to contribute

Yes. Happy to submit a docs PR if the team approves the direction.

---

## 🟡 Recommended Information

### Describe Alternatives You've Considered

Users can manually create ADK `Invocation` objects and run evaluators outside MLflow, but this breaks the unified `mlflow.genai.evaluate()` workflow and loses integration with MLflow's experiment tracking.

### Additional Context

- ADK already has MLflow tracing integration: https://google.github.io/adk-docs/integrations/mlflow/
- MLflow PR adding ADK scorers: https://github.com/mlflow/mlflow/pull/22299
- MLflow issue: https://github.com/mlflow/mlflow/issues/22297
- This follows the pattern of other MLflow third-party scorer integrations (Phoenix/Arize, TruLens/Snowflake, Guardrails AI)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.