Include testing from the source text(s) where possible.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39
- Forks
- 7
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 5
Description
Most teams will have a preference for which source text they wish to use for drafting. After we've run alignments, those can be used to inform the team. Some teams will request drafts from more than one source text.
It would be ideal to include a test set in each training run so that we can obtain metrics that more closely reflect the actual performance of the model. These scores will also make for better comparisons between disparate models.
For example, we might train three models, one from a BT, one from a published Bible and one with both (mixed_src). If all three of those contained a test set corpus pair:
corpus_pairs:
- corpus_books: a small book, or few chapters
src:- src_iso-published_bible
trg: trg_iso-target_bible
type: test
- src_iso-published_bible
It may well be that this is more-or-less already possible and the only change required is to the default config file.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the default config file and the corpus_pairs configuration shown in the issue, then trace how training runs consume entries with type: test. Done means training runs can include a test corpus pair for each requested source text and produce metrics that support comparisons between models.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100