Evaluation should include all three metrics in evalMetrics array by default
- Dominant language
- TypeScript
- Stars
- 1k
- Forks
- 267
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 9
Description
Currently, the evaluation request (evalMetrics array) in the Evaluation tab sends only one metric ('tool_trajectory_avg_score') in the request when running an evaluation. Ideally, the request should include all three relevant metrics, allowing for a more comprehensive evaluation process.
**Proposed solution:**
- Update the code to ensure the evalMetrics array is populated with all three metrics before submitting the evaluation request.
- Consider allowing the user to select which metrics to include, or include all by default.
**Impact:**
- More robust evaluation capability.
- Better alignment with expected evaluation standards.
**References:**
- See `evalMetrics` usage in `src/app/components/eval-tab/eval-tab.component.ts`.
Contributor guide
Assessment
This issue has not been assessed yet.