Create drafts that use the project's selected normalization.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39
- Forks
- 7
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 5
Description
Paratext projects support a "Normalization" setting for the character normalization that they use. Most projects use "NFC" normalization, but some recent projects use "NFD" normalization.
Drafts currently always use the NFKC character normalization implemented by the Sentence Piece tokenizer. Projects that use "NFD" normalization may need to find/replace some NFKC characters in the drafts with the corresponding NFD characters. Enhancing the draft generation workflow to prepare the drafts with their preferred normalization would simplify the use of drafting for these projects.
Flagging the project's normalization setting during the onboarding process could help to highlight this issue for EITL follow up.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the draft generation workflow and how the Sentence Piece tokenizer applies NFKC normalization, then inspect where the project's normalization setting could enter during onboarding. Done means drafts follow the selected project normalization, with the onboarding process highlighting the setting for EITL follow-up if that remains in scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- internationalization, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100