google-research / google-research/google-research

Trouble in running rouge package

Open
#25 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
38.8k
Forks
8.5k
PR merge metrics
No merged PRs in 30d

Description

I was trying to use the rouge package in a multi reference scenario? Is it possible to provide an example in the rouge/readme on how one can do that?
In the current example in "How to run" where do you define the directory of the files?

Also, I am wondering how do you compute rouge for multi-reference scenario. Looking into the code you do some sort of bootstrap aggregation while in the original paper here (https://www.aclweb.org/anthology/W04-1013) in section 2 looks like they do simple micro-averaging while in section 2.1 it seems they do maximization over pairewise computation in rouge!

Thanks,
MP

Contributor guide

Open the contributing guide

Research direction

Start with rouge/readme and the current “How to run” example, then inspect the package code that performs multi-reference aggregation. Compare that behavior with sections 2 and 2.1 of the cited paper. Done means the README shows how to define the file directory, provides a multi-reference example, and explains the computation method.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.