ProjectTech4DevAI / ProjectTech4DevAI/kaapi-backend

Evaluation: STT Config

Open
#595 0 comments 0 reactions 1 assignee View on GitHub

@Prajna1999 is already working on this.

Since Feb 9, 2026.

Dominant language
Python
Stars
18
Forks
10
Avg merge
2d 20h
Merged PRs (30d)
14

Description

Is your feature request related to a problem?

In the first cut of STT evaluation, runs are created with hardcoded parameters (run_name, dataset_id, models). There is no way to pass a standardized config, making it difficult to manage and reproduce evaluation runs with different configurations (e.g. model settings, evaluation parameters). Text evaluation already solves this with config_id + config_version.

Describe the solution you'd like

Add config support to STT evaluation, following the text evaluation pattern:

  • Accept config_id and config_version as input parameters in STTEvaluationRunCreate
  • Resolve the config blob using ConfigVersionCrud before starting the evaluation batch
  • Store config_id and config_version in the EvaluationRun record
  • Extract STT-specific parameters (models, evaluation settings, etc.) from the resolved config

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.