Add MLflow evaluation scorer documentation for ADK evaluators
- Dominant language
- Shell
- Stars
- 1.5k
- Forks
- 1.3k
- Avg merge
- 7d 1h
- Merged PRs (30d)
- 34
Description
## 🔴 Required Information
### Is your feature request related to a specific problem?
ADK's evaluation criteria (ToolTrajectory, ResponseMatch, Hallucinations, etc.) are powerful but users who also use MLflow for experiment tracking have no way to run ADK evaluators through `mlflow.genai.evaluate()`. MLflow already has ADK tracing integration but no evaluation scorer integration.
### Describe the Solution You'd Like
Add a documentation page to ADK docs showing how to use ADK evaluators as MLflow scorers. An MLflow integration has been submitted as PR [mlflow/mlflow#22299](https://github.com/mlflow/mlflow/pull/22299) that wraps ADK's `TrajectoryEvaluator` and `RougeEvaluator` as MLflow third-party scorers.
Example usage:
```python
from mlflow.genai.scorers.google_adk import ToolTrajectory, ResponseMatch
results = mlflow.genai.evaluate(
data=eval_dataset,
scorers=[
ToolTrajectory(match_type="EXACT", threshold=0.5),
ResponseMatch(threshold=0.6),
],
)
```
A docs page under `docs/evaluate/` or `docs/integrations/` showing this integration would help ADK users who track experiments with MLflow.
### Impact on your work
This enables ADK users to evaluate agents through MLflow's unified evaluation pipeline, combining ADK's deterministic evaluators with MLflow's experiment tracking, tracing, and comparison tools.
### Willingness to contribute
Yes. Happy to submit a docs PR if the team approves the direction.
---
## 🟡 Recommended Information
### Describe Alternatives You've Considered
Users can manually create ADK `Invocation` objects and run evaluators outside MLflow, but this breaks the unified `mlflow.genai.evaluate()` workflow and loses integration with MLflow's experiment tracking.
### Additional Context
- ADK already has MLflow tracing integration: https://google.github.io/adk-docs/integrations/mlflow/
- MLflow PR adding ADK scorers: https://github.com/mlflow/mlflow/pull/22299
- MLflow issue: https://github.com/mlflow/mlflow/issues/22297
- This follows the pattern of other MLflow third-party scorer integrations (Phoenix/Arize, TruLens/Snowflake, Guardrails AI)
Contributor guide
Assessment
This issue has not been assessed yet.