kubernetes-sigs / kubernetes-sigs/devops-bench
Determine what type of eval framework we should use
Open
- Dominant language
- Python
- Stars
- 29
- Forks
- 15
- Avg merge
- 6d 3h
- Merged PRs (30d)
- 15
Description
There are many skill eval frameworks in the community such as https://github.com/promptfoo/promptfoo. I wonder if we are going to pick one framework or make the evel in this repo work for mulitple framewok?
Contributor guide
Research direction
Review how evaluation currently works in this repository, then compare its needs with the promptfoo framework linked in the issue. Determine whether to adopt one framework or support multiple frameworks; done means the framework direction and its scope are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100