ProjectTech4DevAI / ProjectTech4DevAI/kaapi-frontend
Evaluation: Update improve-prompt to v2
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 0
- Avg merge
- 10h 40m
- Merged PRs (30d)
- 4
Description
Is your feature request related to a problem?
The improve-prompt currently uses v1, which relies on cosine scores. This limits the ability to provide users with a more comprehensive recommendation based on better metrics available in v2.
Describe the solution you'd like
- Redirect
app/api/evaluations/[id]/improve-prompt/route.tstoPOST /api/v2/evaluations/{evaluation_id}/improve-prompt - Maintain existing
callback_urlresolution and/api/webhooks/prompt-improvementreceiver with unchanged body shape - Implement handling for
PromptRecommendationJobPubliccallback:recommendation_type: "prompt" - Restrict action to judged runs; return
422 not_a_judge_runfor v1 runs
Original issue
Is your feature request related to a problem?
Improve-prompt proxies to v1, which drafts from cosine scores. v2 drafts from the three-metric judge trace (score + reasoning per metric) and returns a typed recommendation, so the console gets a better prompt and a reason for it.
Describe the solution you'd like
- Point
app/api/evaluations/[id]/improve-prompt/route.tsatPOST /api/v2/evaluations/{evaluation_id}/improve-prompt - Keep the existing
callback_urlresolution and/api/webhooks/prompt-improvementreceiver — the body shape is unchanged - Handle the
PromptRecommendationJobPubliccallback:recommendation_type: "prompt"(widening to a union later) - Only offer the action on judged runs; surface
422 not_a_judge_runif it's attempted on a v1 run
Parent: #265
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with app/api/evaluations/[id]/improve-prompt/route.ts and trace its callback_url resolution, then inspect /api/webhooks/prompt-improvement and the available PromptRecommendationJobPublic shape. Verify the judged-run guard and v2 request, preserve the callback body shape, handle recommendation_type: "prompt", and confirm v1 runs return 422 not_a_judge_run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100