google-research / google-research/language
totto bleurt evaluation -- missing flag definition
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 362
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I tried running the bash totto/totto_eval.sh command in the README of the totto repo, but was getting the error:
```
Traceback (most recent call last):
File "~/.conda/envs/totto/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "~/.conda/envs/totto/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "~/fact/language_repo/language/totto/totto_bleurt_eval.py", line 138, in
"bleurt_checkpoint",
File "~/.conda/envs/totto/lib/python3.7/site-packages/absl/flags/_validators.py", line 233, in mark_flags_as_required
mark_flag_as_required(flag_name, flag_values)
File "~/.conda/envs/totto/lib/python3.7/site-packages/absl/flags/_validators.py", line 202, in mark_flag_as_required
if flag_values[flag_name].default is not None:
File "~/.conda/envs/totto/lib/python3.7/site-packages/absl/flags/_flagvalues.py", line 471, in __getitem__
return self._flags()[name]
KeyError: 'bleurt_checkpoint'
```
Adding flags.DEFINE_string("bleurt_checkpoint", None, "") to the totto_bleurt_eval.py file fixes this for me. I also had to down-version sacrebleu to 1.5.1 to get this to work (eval_requirements.txt has no package versions).
Contributor guide
Assessment
This issue has not been assessed yet.