sillsdev / sillsdev/interlinearizer-extension
FieldWorks enriches word glosses but never morpheme glosses, and imported entries lose their FieldWorks provenance
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 2
- Forks
- 0
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 55
Description
Scoped down after testing: a FieldWorks-linked project does import, and ids stay Type:Form-shaped. What is lost is morpheme-level gloss data that FieldWorks holds, and any record of where the mirrored data came from.
Established (Paratext 9.5.110.9, ParatextData 9.5.0.24)
-
The Interlinearizer uses the link.
Paratext.Interlinear.InterlinearForm.LoadInterlinearizercallsIntegratedLexicalProvider.GetLexicon(project, allowFlexData: true)andParatextLexicalAnalyser.Get(project, true); the model text getsfalse. No Paratext plugin is required as of PT9 9.4 — the integration is built in, so an emptyParatext 9\pluginsfolder means nothing. -
The merge is additive.
LexiconWithFlexAdditionsholds bothparatextLexiconandflexLexicon, andEncodesKeyInIddelegates to the Paratext half. -
Word-level selections are mirrored into the file. A FieldWorks entry for "from" produced a plain PT9 entry with a PT9-minted sense id and the FieldWorks gloss copied in:
<Lexeme Type="Word" Form="from" Homograph="1" /> <Entry><Sense Id="WYOx2yJ0"><Gloss Language="es">desda</Gloss></Sense></Entry>referenced as
<Lexeme Id="Word:from" GlossId="WYOx2yJ0" />. SoparseLexemeKeyIdhandles it, our resolver finds entry and sense, and the gloss imports. The "opaque FieldWorks ids break every cluster" risk is not real. -
Morpheme-level lookups never match FieldWorks. In the same project I created a FLEx suffix
-ants(glossagente) and stemserv(glossservir), restarted PT9, and analyzed "servants" asserv ants. PT9 wrote both morphemes asType="Stem"with noGlossIdat all:<Cluster><Range Index="23" Length="8" /><Lexeme Id="Stem:serv" /><Lexeme Id="Stem:ants" /></Cluster>FieldWorks stores
fromandservwith the identical morph-type guid, so its data does not distinguish them. The difference is the lookup: lexeme identity isType+Form+Homograph, the Interlinearizer resolves via the merged lexicon's key indexer (LexiconV2.get_Item, fromInterlinearLexemeClusterCreator.Createand the sense guessers), and FieldWorks entries appear to be exposed asWord-typed — soWord:frommatched whileStem:servdid not. Inferred from the asymmetry, not traced through the provider's type mapping; a PT9 restart ruled out stale caching.
What is wrong for us
- Imported morphemes are unglossed even when FieldWorks has their glosses. Nothing in the files carries
serviroragente, so no import can recover them. The user sees blank morpheme glosses in a project whose lexicon is full. We cannot fix this in PT9, but we can explain it, and a future FW Lite / Lexicon-extension lookup could fill it. - No provenance. A mirrored entry is indistinguishable from a hand-authored one; nothing records that its authority is a FieldWorks entry. #227/#228 refs built from imported data will point at PT9's copy, and #229 cannot tell that an entry already exists upstream.
hasAssociatedLexicalProjectis ignored. It appears in this repo only as a test-fixture default (convertPt9Project.test.ts:19). At minimum it should tell the user the lexicon has an external source of truth — which is exactly the explanation for (1).- Unused FieldWorks inventory is invisible — entries never used in the Interlinearizer never reach the files. Expected, but it bounds what an import can offer for suggestions.
Open question
Are FieldWorks-only word analyses mirrored? WordAnalysesWithFlexAdditions merges them for PT9's own use. WordAnalyses.xml gained nothing from the tests above, but neither test created a FLEx-side analysis, so this is untested rather than answered.
Size: S for (3), M for (2) — it depends on how refs carry provider identity per #225.
Priority: P2 — nothing is corrupted; it bounds the lexicon-ref work and explains a confusing gap to users.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with convertPt9Project.test.ts at line 19, then trace the provider-identity work referenced by #225 and how imported entries and refs are represented. Define and test the handling of hasAssociatedLexicalProject and FieldWorks provenance, while documenting the unresolved morpheme and word-analysis cases; done when the chosen behavior is covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100