ProjectTech4DevAI / ProjectTech4DevAI/kaapi-backend
Evaluation: Tracking overall latency
Open
@AkhileshNegi is already working on this.
Since Jul 10, 2026.
- Dominant language
- Python
- Stars
- 18
- Forks
- 10
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 14
Description
Is your feature request related to a problem?
We lack visibility on the overall evaluation process latency, which affects our ability to identify bottlenecks and optimize performance.
Describe the solution you'd like
- Track overall evaluation latency
- Breakdown latency for cosine score and LLM judge score
- Identify latency added by kaapi
- Explore scope for improvement in latency
Original issue
Tracking below for fast evals (observability)
- Tracking Evals overall latency
- Breakup for latency for cosine score, latency for llm judge score
- Latency added by kaapi ?
Basis above, explore scope for improvement on the latency
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.