dotnet / dotnet/extensions

[AI Evaluation] How should the AI.Eval library be used for runtime AI evaluation?

Open
#6,343 2 comments 1 reaction 0 assignees View on GitHub
area-ai-eval
Dominant language
C#
Stars
3.2k
Forks
894
Avg merge
1d 12h
Merged PRs (30d)
23

Description

All of the samples I've seen are just one-off test methods. Can we have an explicit sample showing how a real app might use the library?

For example, if I have a chat agent that a customer is talking to, how should I evaluate the conversation? Run evaluate after every AI response with the same iteration id so the full conversation will eventually be evaluated and previous sub-conversation evaluations will be overwritten?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.