sillsdev / sillsdev/silnlp

Error on extract_corpora when terms are pulled from another project

Open
#598 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
39
Forks
7
Avg merge
1d 9h
Merged PRs (30d)
5

Description

The project LEDU8 has a seemingly unusual setting for key terms, but one that is apparently allowed by Paratext. It causes extract_corpora.py to inappropriately search for a file that doesn't exist, and then error. It's likely not possible to extract terms in this case, but it would be good to check for this situation and give a warning rather than to fail looking for the inappropriately constructed filename OtherProject-metadata.txt. This is low-priority because (1) it's an infrequent occurrence and (2) the project text still extracts correctly; we just don't get the terms.

Settings file terms configuration:
<BiblicalTermsListSetting>OtherProject:LEDU8:Hughes(Id)BiblicalTerms.xml</BiblicalTermsListSetting>

Error thrown by extract_corpora.py:
OSError: unable to access bucket: 'silnlp' key: 'MT/terms/OtherProject-metadata.txt' version: None error: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with extract_corpora.py and trace how the BiblicalTermsListSetting value is converted into the terms metadata filename. Reproduce the LEDU8 configuration if possible and verify that extraction warns about the cross-project terms instead of failing on OtherProject-metadata.txt, while the project text still extracts correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.