[Bug] output/expected arguments swapped in LLMJudge - evaluation correctness inverted

Bug Description

In highlight_summarize/judges.py, the LLMJudge.__call__ method passes arguments to call_judge in the wrong order:

# Current (incorrect)
judgement = self.call_judge(
    input=example["question"],
    output=example["answer"],        # ground truth passed as model output
    expected=example["answer_pred"], # model prediction passed as expected
)

Problem

The judges library follows the convention:

• output → model prediction
• expected → ground truth

However, the current implementation reverses these, causing:

• Incorrect evaluation logic
• Inverted correctness scores (especially for asymmetric judges)
• Misleading benchmarking results

---

Proposed Fix

# Corrected implementation
judgement = self.call_judge(
    input=example["question"],
    output=example["answer_pred"],  # model prediction
    expected=example["answer"],     # ground truth
)

---

Additional Issue

The constructor uses a mutable default argument:

def __init__(self, judge_name, correct=10, incorrect=1, factors=[]):

Problem

Using a mutable default ([]) can lead to unintended shared state across instances.

---

Fix for Constructor

def __init__(self, judge_name, correct=10, incorrect=1, factors=None):
    self.factors = factors if factors is not None else []

---

How to Verify

1. Navigate to highlight_summarize/judges.py (around line ~65).
2. Confirm the argument order in call_judge.
3. Cross-check with judges library documentation:
   • output = model prediction
   • expected = reference/ground truth

---

Expected Outcome

• Correct evaluation scores
• Proper alignment with judges library conventions
• Elimination of misleading metrics

---

Environment

• Python: 3.x
• Package version: main branch (commit 6550782)

---

Impact

This bug affects all evaluations using LLMJudge and may silently produce incorrect benchmarking results.

Start in highlight_summarize/judges.py, focusing on LLMJudge.call around line 65 and the constructor's factors parameter. Cross-check the call_judge argument convention with the judges library documentation and inspect how factors is stored. Done means predictions and ground truth are passed to the intended arguments and separate instances do not share mutable default state.