sillsdev / sillsdev/silnlp

Confidence feature not working on test.py for --multiple-translations

Open
#911 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug pipeline 5: test
Dominant language
Python
Stars
39
Forks
7
Avg merge
1d 9h
Merged PRs (30d)
5

Description

When the --multiple-translations flag is set for test.py, the following error occurs:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/experiment.py", line 301, in <module>
    main()
  File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/experiment.py", line 297, in main
    exp.run()
  File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/experiment.py", line 61, in run
    self.test()
  File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/experiment.py", line 85, in test
    test(
  File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/test.py", line 827, in test
    results[step] = test_checkpoint(
  File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/test.py", line 664, in test_checkpoint
    pair_sys, pair_refs, book_dict = load_test_data(
  File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/test.py", line 507, in load_test_data
    book_dict = process_individual_books(
  File "/root/.clearml/venvs-builds/3.10/task_repository/silnlp.git/silnlp/nmt/test.py", line 402, in process_individual_books
    conf_file = stack.enter_context(conf_file_path.open("r", encoding="utf-8"))
  File "/usr/lib/python3.10/pathlib.py", line 1119, in open
    return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/root/M/MT/experiments/Demo_Matthew/test_translate_NT_1/test.trg-predictions.txt.5000.confidences.tsv'

Seems like this is due to some issue with the path name handling for the confidence file in this scenario.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in silnlp/nmt/test.py, tracing test_checkpoint through load_test_data to process_individual_books, where the confidence file is opened. Reproduce test.py with --multiple-translations and inspect how the confidence-file path is constructed. Done means the confidence file is found and the test completes without the reported FileNotFoundError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.