globalwordnet / globalwordnet/semcor

Spurious gap between an abbreviation period and the sentence-final period (Jr. . instead of Jr..)

Open
#61 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
Avg merge
2d 22h
Merged PRs (30d)
24

Description

When a sentence ends right after an abbreviation (\`Jr.\`, \`etc.\`, \`D.C.\`, ...), Brown's own convention keeps both the abbreviation's period and the sentence-final period as two adjacent, flush characters (\`Jr..\`) rather than eliding one -- confirmed directly against \`src/semcor/brown-nolines.txt\` (e.g. \`Mayor-nominate Ivan Allen Jr..\`). This corpus's own data has 154 raw candidates of an abbreviation-ending token immediately followed by a separate \`.\` token with a single spurious space between them (\`Jr. .\`).

Context-window verification against \`brown-nolines.txt\` (same technique as the #8 quote-gap follow-up and #43's hyphen-dropped-word-pair fix) confirms 100 of these unambiguously. The other 54 are left unresolved for now, mostly because the abbreviation itself is a multi-word underscore-joined token (\`N._Y.\`, \`D._C.\`, \`U._S.\`) where Brown's real text has *no* space at all between the parts either (\`N.Y..\`, not \`N. Y..\`) -- a related but distinct bug in how this corpus's underscore-joining convention handles this class of abbreviation, worth its own separate look rather than folding it into this fix.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading src/semcor/brown-nolines.txt and compare the 100 unambiguous abbreviation-period candidates against the Brown source context. Trace the repository's corpus correction or generation workflow before editing, and verify that those cases use adjacent periods while the 54 unresolved multi-word abbreviation cases remain out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.