sillsdev / sillsdev/silnlp

Translate script doesn't support USFM file names using the "<nn>-<book>" naming format

Open
#456 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement pipeline 6: infer
Dominant language
Python
Stars
39
Forks
7
Avg merge
1d 9h
Merged PRs (30d)
5

Description

Paratext-compatible projects downloaded from Door43 (e.g., https://git.door43.org/unfoldingWord/el-x-koine_ugnt) use the "-" book name format for naming the USFM files in the project. For example:

  • 01-GEN.usfm
  • 02-EXO.usfm
  • 03-LEV.usfm
  • ...

When the translate script is run with one of these projects as the source projects, the script will error out because it doesn't properly handle this book naming format:

2024-07-13 07:58:52,216 - silnlp.nmt.translate - ERROR - Was not able to translate MIC.
Traceback (most recent call last):
  File "/root/.clearml/venvs-builds/3.8/task_repository/silnlp.git/silnlp/nmt/translate.py", line 122, in translate_books
    translator.translate_book(
  File "/root/.clearml/venvs-builds/3.8/task_repository/silnlp.git/silnlp/common/translator.py", line 317, in translate_book
    raise RuntimeError(f"Can't find file {book_path} for book {book}")
RuntimeError: Can't find file /tmp/tmpu35zij1b/hbo_uhb_2024_07_10/33MIC.usfm for book MIC

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in silnlp/nmt/translate.py and silnlp/common/translator.py, tracing how the translate script derives each USFM path from a book code. Reproduce the failure with a Paratext-compatible Door43 project containing names such as 01-GEN.usfm, then verify that the nn-book naming format is accepted without the missing-file error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.