gomate-community / gomate-community/rageval

Metrics info need update

Open
#128 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
172
Forks
10
PR merge metrics
No merged PRs in 30d

Description

In the last update, we standardized the input parameters of `metric._compute_one` to use names like 'pred_answer' and 'ref_answers', but the corresponding `feature` section in `metric._info()` was not updated accordingly—it still retains the old naming conventions ('answers' and 'gt_answers').

Additionally, in the newer version of the `datasets` package, the `MetricInfo` class has been deprecated and should be replaced with `evaluate.MetricInfo`.

Contributor guide

Open the contributing guide

Research direction

Locate the metric._compute_one and metric._info() definitions and inspect the feature section for the old parameter names. Check the current dependency usage for datasets.MetricInfo versus evaluate.MetricInfo, then update both references and run the relevant metric tests or evaluation checks to confirm the metadata and inputs remain consistent.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Refactor
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.