ProjectTech4DevAI / ProjectTech4DevAI/kaapi-frontend
Evaluation UI: Model Config
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 0
- Avg merge
- 10h 40m
- Merged PRs (30d)
- 4
Description
Is your feature request related to a problem?
The current evaluation config only supports basic options: model name, vector store ID, and system prompt. However, different models have different configuration parameters—newer models support reasoning and verbosity, while older models use temperature and top_p.
This limits our ability to properly evaluate models with their optimal settings.
Describe the solution you'd like
Add support for model-specific configuration options in the evaluation config:
- Newer models: reasoning, verbosity
- Older models: temperature, top_p, max_tokens
This would:
- Allow evaluations with model-appropriate parameters
- Enable testing different config combinations (e.g., high vs low temperature)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files or tests are named. Start by locating the evaluation configuration UI and the code that builds model configuration, then trace how the current model name, vector store ID, and system prompt are handled. Done means model-specific reasoning, verbosity, temperature, top_p, and max_tokens options can be configured for evaluations, including different combinations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- frontend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100