alibaba / alibaba/ROLL

LLM_judge question

Open
#106 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.4k
Forks
312
Avg merge
1h 2m
Merged PRs (30d)
2

Description

If I want to use qwen3 embedding to generate and ground truth to calculate the similarity after embedding as a reward, does this need to be modified to the llm_judge format?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the existing llm_judge format and the reward paths that handle embeddings and ground truth similarity. Compare their inputs and outputs to determine whether this use case fits the current format or requires a new integration. Done should be a decided scope with the relevant implementation and validation points identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.