sillsdev / sillsdev/silnlp

Research: Two source languages - train and inference off of both

Open
#295 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs to be revisited research
Dominant language
Python
Stars
39
Forks
7
Avg merge
1d 9h
Merged PRs (30d)
5

Description

Spin off from: https://github.com/sillsdev/serval/issues/265

Different source languages for the different drafts.

  • Train an xx/yy translation model, then translate book A from the source translation (xx) into the target language.
  • Train a zz/yy translation model, then translate book A from the source translation (zz) into the target language.

Can we train one model to translate from both sources to one target? That is what NLLB does - how can we do it best? Would it be better just to train 2 models in Serval and switch between them?

This could also ask the question about multiple source texts from the same language, possibly with tags for different translations, etc. It is part of a broader discussion about multiple sources and potentially multiple drafts for the target for a given verse.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the linked silnlp issue #265 and the NLLB approach referenced in this issue. Compare a shared model for two source languages with training two separate models, including the question of multiple source texts or translation tags. Done means a documented design decision with enough scope to guide implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.