elastic / elastic/relevance-studio
[Feature]: Batch API endpoint for creating judgements
- Dominant language
- Python
- Stars
- 42
- Forks
- 18
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 4
Description
### Problem Statement
When creating multiple judgements, especially using an LLM and MCP, sending individual create judgement requests becomes inefficient.
### Proposed Solution
This feature would add a batch create to the [judgements API](https://elastic.github.io/relevance-studio/#/docs/latest/reference/rest-api?id=judgements-api).
One solution would be to allow an alternate payload format in the existing endpoint, e.g. :
```
{
"judgements": [
{
"scenario_id": "67d554e2-8622-4c6c-9601-bf0e0a06128d",
"index": "demo",
"doc_id": "123",
"rating": 3
}
]
}
```
### Alternatives Considered
_No response_
### Enhancement Area
Frontend (UI)
### Priority
Critical - Blocking my workflow
### Use Case
_No response_
### Additional Context
_No response_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the judgements API documentation and the existing create-judgement endpoint. Trace its request validation and tests, then determine how the proposed judgements array should be accepted and verified. Done means one request creates all supplied judgements with the documented payload and coverage for the batch behavior.
Written by the indexing model from the issue text.
Assessment
- Domain
- api, frontend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100