sillsdev / sillsdev/interlinearizer-extension

Strip blank morpheme forms on import and assert the invariant

Open
#324 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
2
Forks
0
Avg merge
1d 17h
Merged PRs (30d)
55

Description

Per the decision on #285, blank morpheme forms are dropped at import rather than carried. This issue owns that change and the invariant it establishes.

What arrives today

PT9 files can carry a lexeme id with an empty form, and every check on our side counts keys rather than inspecting them, so the records reach the store intact. Observed in one small test project:

surface morphemes source
thanksgiving ["", "giving"] WordAnalyses.xml
thanksgiving ["", ""] WordAnalyses.xml
prayer [""] WordAnalyses.xml, from the id Stem::2

The import reported droppedEmpty: 0 and droppedUnparseable: 0 for all three. A cluster can carry them too (["Stem:", "Stem:giving"]), though in that project the cluster dropped for an unrelated reason.

What to change

Two conversion boundaries, each with a visible counter:

  • bareWordAnalyses.ts — its only rejection is keys.length === 0 (:54), which counts keys rather than inspecting them.
  • The cluster path — clusterAnchoring.ts classifies lexemes without looking at the form; a new drop reason belongs alongside unparseableLexemeId.

Leave parseLexemeKeyId alone. It is faithful to PT9's grammar, which admits Type: with an empty form, so rejecting there would make the parser diverge from what its doc comment claims to implement.

The one rule to decide

An all-blank parse drops cleanly — it states nothing. A mixed parse is the judgment call:

  • Drop only the blank morpheme, leaving ["giving"] for "thanksgiving" — but that fabricates a breakdown whose forms do not sum to the surface, which is #317's problem class, and worse because we would be creating it rather than importing it.
  • Drop the whole parse, which is more defensible and keeps the store free of breakdowns we would immediately flag.

Decide this together with #317, since both concern breakdowns that do not sum to their surface.

The invariant this establishes

Nothing in the store carries form === "" afterwards, and our editor cannot create one either — MorphemeEditor collapses and trims (MorphemeEditor.tsx:112) and TokenChip filters (:225). Make that explicit so a future import path or edit affordance cannot reintroduce blanks silently:

  • Assert it in the validation pass proposed by #140, alongside the invariants already listed there.
  • Keep the counters in the import report, so a project whose PT9 data contains zero morphemes says so rather than quietly shedding them.

Extensions are pre-release, so already-imported records need no migration.

What this makes unnecessary

Rendering work for blank forms. MorphemeBox.tsx:106 puts {m.form} straight into a grid cell and breakdownOf joins forms on a space, so both would need a placeholder glyph, a spoken label, and a #130-compatible normalization — three surfaces of upkeep for a state that can no longer occur. #130 also no longer needs a blank-form rule.

What is being given up

PT9 treats a zero morpheme as ordinary data: it renders one as an empty // slot in both the Interlinearizer and the Wordlist, auto-selects its sense as the gloss, lets that gloss be edited, and preserves it across its own saves. Stripping trades that away for a counter. #285 records the evidence should the decision ever be revisited.

Size: S.
Priority: P2 — no user-visible defect today, but it is the gate on #313's fold and on closing #285.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

First resolve the mixed-parse decision with #317. Then inspect bareWordAnalyses.ts and clusterAnchoring.ts, preserving parseLexemeKeyId, and trace the validation pass proposed in #140. Done means blank forms cannot reach the store or editor, the import report counts dropped data, and the invariant is asserted.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.