sillsdev / sillsdev/silnlp

Improved mixing of back translation in drafting configuration

Open
#837 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement pipeline 3: preprocess
Dominant language
Python
Stars
39
Forks
7
Avg merge
1d 9h
Merged PRs (30d)
5

Description

During preprocessing, the current mixed source logic does a random 50/50 split of the main reference translation and the back translation to create the source side of the training dataset. That is, 50% of the source verses are taken from the main reference translation, and the other 50% of the verses are taken from the back translation. This is not likely to be the best way to get value out of the back translation project.

Most of the time, the back translation will be the most valuable for verses where the target verse is least similar to the source verse. This lack of similarity is likely because the project has needed to explicate the verse in more detail or express the idea very differently for various reasons. For these verses, the project's back translation of the verse is likely to be a better representation of the project's verse translation. Pairing the back translation text as the source text for this verse is likely to be a better training setup. Altering the mixed source logic to have, e.g., an alignment-based score threshold, and only using the BT verse when the score threshold is not met, would ensure that the back translation text is only used when it's most likely to be helpful.

I believe that the mixed source logic is applied to the train, validation, and test sets. If that's the case, another improvement would be to limit the mixing to just the training set. There's no need to mix the BT into the validation and test sets. When the model is actually used to create a draft, it will only be used on the reference translation, and the best validation / test set would consist of only the reference translation verses.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names only the preprocessing mixed-source logic, not a file, test, or entry point. Start by locating that logic and verify which dataset splits it affects. Then investigate available alignment scoring and define the threshold behavior; done requires an agreed selection rule and evidence that validation and test inputs remain reference-only.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.