globalwordnet / globalwordnet/semcor

Brace-casing-adoption missed every interior word of a multi-word {...} span

Open
#66 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
Avg merge
2d 22h
Merged PRs (30d)
24

Description

Spotted via a leftover \`-may\`/\`+MAY\` in \`brown-nolines.diff\` right at
\`{DALLAS MAY GET} to hear a debate on horse race parimutuels...\` --
this exact example was the original motivating case for the \`{...}\`
casing-adoption mechanism, but \`MAY\` itself was never actually being
adopted.

Root cause in \`_decode_and_split\`: a word only carries a literal \`{\`/\`}\`
character when it's flush against the brace (\`{DALLAS\`, \`GET}\`) -- a
middle word of a multi-word span (\`MAY\`) has neither character, so
flagging word-by-word (\`"{" in w or "}" in w\`) missed every interior
word of every span longer than one word. \`DALLAS\` and \`GET\` (the
words actually touching a brace) were being correctly adopted; only the
words *between* them never were.

Fixed by tracking a running "currently inside a span" state instead, so
every word from the opening \`{\` through the closing \`}\` (inclusive)
gets flagged -- capped at 50 words so a brace that never closes
(brown_nolines.txt has a few more \`}\` than \`{\`) doesn't mis-flag the
rest of the document. Brace-derived word count found: 462 -> ~1,600.
Divergent word-line count: 16660 -> 16562, no \`data/*.yaml\` changes.

A separate, harder case remains unhandled: the same ALL-CAPS
paragraph-lead convention sometimes appears with *no* brace wrapper at
all (\`YOU MAY DO AS YOU PLEASE with God now.\`), which has no signal to
key off.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at _decode_and_split and inspect how brace-delimited words are represented, then compare the behavior against the example in brown-nolines.diff and the related brown_nolines.txt data. Done means interior words in multi-word brace spans are flagged without extending past an unclosed span; verify the reported word-line counts and that data/*.yaml is unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.