ProjectTech4DevAI / ProjectTech4DevAI/kaapi-frontend

Evaluation UI: Model Config

Open
#8 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
TypeScript
Stars
1
Forks
0
Avg merge
10h 40m
Merged PRs (30d)
4

Description

Is your feature request related to a problem?

The current evaluation config only supports basic options: model name, vector store ID, and system prompt. However, different models have different configuration parameters—newer models support reasoning and verbosity, while older models use temperature and top_p.
This limits our ability to properly evaluate models with their optimal settings.

Describe the solution you'd like

Add support for model-specific configuration options in the evaluation config:

  • Newer models: reasoning, verbosity
  • Older models: temperature, top_p, max_tokens

This would:

  • Allow evaluations with model-appropriate parameters
  • Enable testing different config combinations (e.g., high vs low temperature)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by locating the evaluation configuration UI and the code that builds model configuration, then trace how the current model name, vector store ID, and system prompt are handled. Done means model-specific reasoning, verbosity, temperature, top_p, and max_tokens options can be configured for evaluations, including different combinations.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
frontend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.