google-research / google-research/language

totto bleurt evaluation -- missing flag definition

Open
#121 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.8k
Forks
362
PR merge metrics
No merged PRs in 30d

Description

Hi, I tried running the bash totto/totto_eval.sh command in the README of the totto repo, but was getting the error:
```
Traceback (most recent call last):
File "~/.conda/envs/totto/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "~/.conda/envs/totto/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "~/fact/language_repo/language/totto/totto_bleurt_eval.py", line 138, in
"bleurt_checkpoint",
File "~/.conda/envs/totto/lib/python3.7/site-packages/absl/flags/_validators.py", line 233, in mark_flags_as_required
mark_flag_as_required(flag_name, flag_values)
File "~/.conda/envs/totto/lib/python3.7/site-packages/absl/flags/_validators.py", line 202, in mark_flag_as_required
if flag_values[flag_name].default is not None:
File "~/.conda/envs/totto/lib/python3.7/site-packages/absl/flags/_flagvalues.py", line 471, in __getitem__
return self._flags()[name]
KeyError: 'bleurt_checkpoint'
```
Adding flags.DEFINE_string("bleurt_checkpoint", None, "") to the totto_bleurt_eval.py file fixes this for me. I also had to down-version sacrebleu to 1.5.1 to get this to work (eval_requirements.txt has no package versions).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.