sillsdev / sillsdev/silnlp

Add a mixed_trg option for corpus pairs

Open
#766 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement pipeline 3: preprocess
Dominant language
Python
Stars
39
Forks
7
Avg merge
1d 9h
Merged PRs (30d)
5

Description

Some projects that don't make a practice of creating back translations. When there's a need to review their vernacular text with an outsider (e.g., a translation consultant), the effort to create a back translation is challenging. And, since the team doesn't have an existing base of back translation text, these projects are not good candidates for current drafting techniques.

As a method for bootstrapping a new back translation of a book X, existing published translations in the back translation language can be used. The model would be trained by pairing books different from the vernacular project with the corresponding books from one of the published translations. Then, the trained model would be used to translate book X into the back translation language.

Ideally, the books of the vernacular translation would be evenly spread across multiple published translations, so that the back translation does not follow a specific published translation too closely. The current book-by-book method of distributing the books among the published translations is cumbersome and likely less optimal. A random distribution, similar to the mixed_src option for randomly selecting from multiple sources, would be preferred.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the existing mixed_src option and the corpus-pair configuration or entry point that handles source selection. Use that behavior to determine where a mixed_trg option belongs, then verify that books are randomly distributed across the available published translations rather than assigned book by book.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.