Confidence feature not working on test.py for --multiple-translations
Open
Nobody has claimed this yet.
bug
pipeline 5: test
- Dominant language
- Python
- Stars
- 39
- Forks
- 7
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 5
Description
When the --multiple-translations flag is set for test.py, the following error occurs:
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/experiment.py", line 301, in <module>
main()
File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/experiment.py", line 297, in main
exp.run()
File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/experiment.py", line 61, in run
self.test()
File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/experiment.py", line 85, in test
test(
File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/test.py", line 827, in test
results[step] = test_checkpoint(
File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/test.py", line 664, in test_checkpoint
pair_sys, pair_refs, book_dict = load_test_data(
File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/test.py", line 507, in load_test_data
book_dict = process_individual_books(
File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/test.py", line 402, in process_individual_books
conf_file = stack.enter_context(conf_file_path.open("r", encoding="utf-8"))
File "/usr/lib/python3.10/pathlib.py", line 1119, in open
return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/root/M/MT/experiments/Demo_Matthew/test_translate_NT_1/test.trg-predictions.txt.5000.confidences.tsv'
Seems like this is due to some issue with the path name handling for the confidence file in this scenario.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in silnlp/nmt/test.py, tracing test_checkpoint through load_test_data to process_individual_books, where the confidence file is opened. Reproduce test.py with --multiple-translations and inspect how the confidence-file path is constructed. Done means the confidence file is found and the test completes without the reported FileNotFoundError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100