sillsdev / sillsdev/machine

HermitCrab: unhandled NullReferenceException in Morpher.ParallelSynthesize on an empty InsertSegments PhoneticShape

Open
#440 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
29
Forks
17
Avg merge
3d 2h
Merged PRs (30d)
8

Description

Unhandled NullReferenceException in Morpher.ParallelSynthesize when a MorphologicalRule's InsertSegments has an empty PhoneticShape

Component: SIL.Machine.Morphology.HermitCrab (HermitCrab morphological parser) — used via the hc CLI (SIL.Machine.HCParser).
Severity: Medium. A grammar-authoring mistake (an empty insertion shape) crashes the whole process with an unhandled exception instead of being reported as a grammar/validation error. The grammar compiles and loads cleanly ("Compiling rules... done", " loaded"); the crash happens at parse time, so it is easy to mistake for a parser bug in otherwise-valid grammars.

Environment

  • sil.machine.hcparser 2.2.3 (the hc .NET global tool), .NET (DOTNET_ROLL_FORWARD=LatestMajor).
  • OS: Windows 11.
  • Build paths in the stack trace: C:\projects\machine\src\SIL.Machine.Morphology.HermitCrab\Morpher.cs.

Summary

A MorphologicalRule (affix process) whose MorphologicalOutput contains
<InsertSegments><PhoneticShape></PhoneticShape></InsertSegments> (an empty phonetic shape) causes
hc.exe to throw an unhandled System.NullReferenceException during synthesis when any word is parsed.
The configuration compiles without error; the exception is only raised at parse time, inside
Morpher.ParallelSynthesize (HC verifies each analysis by re-synthesizing it).

Minimal reproduction

Self-contained grammar (one root tu, one affix rule that inserts an empty shape before the stem):

<?xml version="1.0" encoding="utf-8"?>
<HermitCrabInput><Language><Name>x</Name>
<PartsOfSpeech><PartOfSpeech id="root"><Name>root</Name></PartOfSpeech></PartsOfSpeech>
<PhonologicalFeatureSystem><SymbolicFeature id="seg" defaultSymbol="g0"><Name>seg</Name>
<Symbols><Symbol id="g0">g0</Symbol><Symbol id="g1">g1</Symbol></Symbols></SymbolicFeature>
</PhonologicalFeatureSystem>
<CharacterDefinitionTable id="t1"><Name>m</Name><SegmentDefinitions>
<SegmentDefinition id="s_t"><Representations><Representation>t</Representation></Representations>
<FeatureValue feature="seg" symbolValues="g0"/></SegmentDefinition>
<SegmentDefinition id="s_u"><Representations><Representation>u</Representation></Representations>
<FeatureValue feature="seg" symbolValues="g1"/></SegmentDefinition>
</SegmentDefinitions></CharacterDefinitionTable>
<NaturalClasses><SegmentNaturalClass id="any"><Name>any</Name>
<Segment segment="s_t"/><Segment segment="s_u"/></SegmentNaturalClass></NaturalClasses>
<Strata><Stratum characterDefinitionTable="t1" morphologicalRuleOrder="unordered"
morphologicalRules="afx"><Name>main</Name><MorphologicalRuleDefinitions>
<MorphologicalRule id="afx" requiredPartsOfSpeech="root" outputPartOfSpeech="root"><Name>afx</Name>
<MorphologicalSubrules><MorphologicalSubrule id="afxs"><MorphologicalInput>
<PhoneticSequence id="st"><OptionalSegmentSequence min="1" max="-1"><SimpleContext naturalClass="any"/>
</OptionalSegmentSequence></PhoneticSequence></MorphologicalInput><MorphologicalOutput>
<InsertSegments><PhoneticShape></PhoneticShape></InsertSegments>   <!-- EMPTY shape -->
<CopyFromInput index="st"/></MorphologicalOutput></MorphologicalSubrule></MorphologicalSubrules>
<Gloss>afx</Gloss></MorphologicalRule></MorphologicalRuleDefinitions>
<LexicalEntries><LexicalEntry id="e0" partOfSpeech="root"><Allomorphs>
<Allomorph id="e0a"><PhoneticShape>tu</PhoneticShape></Allomorph></Allomorphs><Gloss>tu</Gloss>
</LexicalEntry></LexicalEntries></Stratum></Strata></Language></HermitCrabInput>

Run:

hc -i grammar.xml -s script.txt -o out.txt -c      # script.txt contains:  parse tu

Actual result

Reading configuration file "grammar.xml"... done.
Compiling rules... done.
x loaded.

Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at SIL.Machine.Morphology.HermitCrab.Morpher.ParallelSynthesize(IEnumerable`1 analyses) in C:\projects\machine\src\SIL.Machine.Morphology.HermitCrab\Morpher.cs:line 305
   at SIL.Machine.Morphology.HermitCrab.Morpher.ParseWord(String word, Object& trace) in C:\projects\machine\src\SIL.Machine.Morphology.HermitCrab\Morpher.cs:line 114
   at SIL.Machine.Morphology.HermitCrab.ParseCommand.Run(String[] remainingArguments) in C:\projects\machine\src\SIL.Machine.HCParser\ParseCommand.cs:line 30
   at ManyConsole.ConsoleCommandDispatcher.DispatchCommand(...)
   at SIL.Machine.Morphology.HermitCrab.Program.Main(String[] args) in C:\projects\machine\src\SIL.Machine.HCParser\Program.cs:line 95

The process exits abnormally and writes no output file.

Expected result

Either:

  1. Validation error at load/compile time — e.g. "MorphologicalRule 'afx': InsertSegments has an empty PhoneticShape", with a non-zero but handled exit; or
  2. The empty insertion is treated as a no-op (insert nothing) and parsing proceeds normally.

Crashing with an unhandled NullReferenceException at parse time is the problem regardless of which is chosen.

Notes on how it arises in practice

We hit this when programmatically generating affix-process rules: the inserted PhoneticShape was built from characters that were not all present in the grammar's CharacterDefinitionTable, so the shape encoded to an empty string. The grammar still compiled, then every parse crashed. A clear validation message (or a tolerant no-op) would have turned a hard-to-localize crash into an obvious authoring error. (A related hardening: reject/validate an InsertSegments PhoneticShape containing representations not defined in the character table.)

Suggested fix

Guard Morpher.ParallelSynthesize (Morpher.cs:~305) against an affix process that inserts an empty/!null segment shape, and/or validate non-empty InsertSegments PhoneticShape (with all representations defined in the CharacterDefinitionTable) at rule-compile time.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at Morpher.cs around line 305 and reproduce the crash with the provided grammar and hc -i grammar.xml -s script.txt -o out.txt -c command. Trace how an empty InsertSegments PhoneticShape reaches ParallelSynthesize, then add coverage for the parse case and verify it produces either a handled validation error or a successful no-op instead of an unhandled exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.