globalwordnet / globalwordnet/semcor

Ordinary sentence commas dropped (unrelated to thousands separators)

Open
#31 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
0
Forks
0
Avg merge
2d 22h
Merged PRs (30d)
24

Description

Part of #5. Split off from #15's investigation.

#15 flagged that its own raw count of missing commas included "a smaller
number of ordinary sentence commas dropped for unrelated reasons." Re-running
`semcor-verify-brown`'s whole-document alignment against `nltk.corpus.brown`,
isolating pure single-comma deletions that are *not* between two digits,
confirms **52** such instances, spread across mixed genres (fiction,
learned, press) -- list-item commas, appositive commas, a comma dropped
next to a quote.

Unlike #15's thousands-separator case, there's no single shared trigger
across the sample -- each looks like an independent dropped comma rather
than one systematic transformation. Worth its own triage rather than
folding into #15's fix.

A further 3 instances are a related but distinct anomaly: a whole
`,` sequence (comma *and* digit) missing entirely, e.g. Brown has a
trailing `3,`/`7,`/`4,` that's fully absent from this corpus's `text` --
too rare to investigate on their own, but worth keeping in mind while
triaging the 52 above in case they share a cause.

### Suggested fix

No single mechanical fix suggests itself the way #15's did -- these likely
need per-instance triage against Brown context to confirm each is really a
missing comma (vs. e.g. a legitimate editorial choice) before fixing.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by rerunning `semcor-verify-brown`'s whole-document alignment against `nltk.corpus.brown`, isolating single-comma deletions that are not between digits. Review each of the 52 instances against Brown context, while noting the three missing digit-comma sequences; done means the cases are confirmed as errors or legitimate editorial choices and appropriate corrections are identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.