[AI Evaluation] How should the AI.Eval library be used for runtime AI evaluation?
Open
area-ai-eval
- Dominant language
- C#
- Stars
- 3.2k
- Forks
- 894
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 23
Description
All of the samples I've seen are just one-off test methods. Can we have an explicit sample showing how a real app might use the library?
For example, if I have a chat agent that a customer is talking to, how should I evaluate the conversation? Run evaluate after every AI response with the same iteration id so the full conversation will eventually be evaluated and previous sub-conversation evaluations will be overwritten?
Contributor guide
Assessment
This issue has not been assessed yet.