[Feat]: evals extensions
Open
extension-proposal
triaged
- Dominant language
- Shell
- Stars
- 25.7k
- Forks
- 2.6k
- Avg merge
- 3d 6h
- Merged PRs (30d)
- 16
Description
### Is your feature request related to a problem? Please describe.
I can put example prompts into the card. I'd like to put evals into the card too. This could be an extension. It would make the evals public and conventional meaning the same framework can be used for any agent. You could build you evals once.
I'm not sure what it would look like, but presumably a prompt and some eval criteria, e.g. an expression that matches the output.
### Describe the solution you'd like
An extension.
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Contributor guide
Assessment
This issue has not been assessed yet.