google-research / google-research/google-research
Trouble in running rouge package
- Dominant language
- Jupyter Notebook
- Stars
- 38.8k
- Forks
- 8.5k
- PR merge metrics
- No merged PRs in 30d
Description
I was trying to use the rouge package in a multi reference scenario? Is it possible to provide an example in the rouge/readme on how one can do that?
In the current example in "How to run" where do you define the directory of the files?
Also, I am wondering how do you compute rouge for multi-reference scenario. Looking into the code you do some sort of bootstrap aggregation while in the original paper here (https://www.aclweb.org/anthology/W04-1013) in section 2 looks like they do simple micro-averaging while in section 2.1 it seems they do maximization over pairewise computation in rouge!
Thanks,
MP
Contributor guide
Research direction
Start with rouge/readme and the current “How to run” example, then inspect the package code that performs multi-reference aggregation. Compare that behavior with sections 2 and 2.1 of the cited paper. Done means the README shows how to define the file directory, provides a multi-reference example, and explains the computation method.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100