GoogleCloudPlatform / GoogleCloudPlatform/generative-ai

[Feat]: Add sample of Vertex AI Generative AI Evaluation service with PydanticAI

Open
#1,660 0 comments 0 reactions 1 assignee Claimed by @ghchinoy View on GitHub
Dominant language
Jupyter Notebook
Stars
17.7k
Forks
4.5k
Avg merge
12h 38m
Merged PRs (30d)
42

Description

### Is your feature request related to a problem? Please describe.

I'd like to see a PydanticAI agent evaluated with the Vertex AI Generative AI Evaluation service SDK so that I can utilize VAI Gen AI Evals on my agents.

### Describe the solution you'd like

A python example using the Vertex AI Generative AI Evaluation service SDK and a PydanticAI agent showing single tool use, tool trajectory use, and final response evaluations, similar to the existing notebooks for LangGraph and CrewAI.

### Describe alternatives you've considered

A notebook, though colab's google-auth and PydanticAI's google-auth versions clash (bug filed elsewhere).

### Additional context

_No response_

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.