ByteDance-Seed / ByteDance-Seed/WideSearch

Potential bug in`llm_judge_column`

Open
#9 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
154
Forks
18
PR merge metrics
No merged PRs in 30d

Description

Inside function `llm_judge_column`, it seems that the following code will not be executed, since `len(score_list) == len(response)` in current implementation.
```
if len(score_list) != len(response):
score_list = [0] * len(response)
msg_list = ["llm judge failed due to length"] * len(response)
```
https://github.com/ByteDance-Seed/WideSearch/blob/9825ba7b140b71d81b364793f86dabe4cfed6749/src/evaluation/metric_utils.py#L376

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.