globalwordnet / globalwordnet/semcor
17 more underscore-joined hyphenated lexemes missed by #43, plus one editorial restructure (junior-senior high)
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 24
Description
Re-scanning for #43's own pattern (a token's surface uses \`_\` where its own \`lemmas\` entry already has \`-\` at the same position, e.g. \`ex_gambler\` with lemma \`ex-gambler\`) finds 17 more instances #43's original scan missed, each individually confirmed against \`src/semcor/brown-nolines.txt\`. Added to \`src/semcor/underscore-hyphen-lexeme-fixes.yaml\` (54 -> 71) and fixed via the existing \`semcor-fix-underscore-hyphen-lexemes\`, unchanged.
Of 91 raw candidates (same lemma-vs-surface signal), 74 were correctly left alone: the lemma's hyphen is just WordNet's own dictionary-citation-form convention for the multiword entry (\`such-and-such\`, \`upside-down\`, \`half hour\` in this specific instance, ...) and Brown's real text at that exact position genuinely uses a plain space, not a hyphen -- confirmed per-instance, since Brown is genuinely inconsistent between space and hyphen for some of these across different occurrences (e.g. \`half hour\`/\`half-hour\` both appear elsewhere).
Separately, \`data/press_reportage/br-a02.yaml\`'s \`qwwa\` (\`junior senior_high teachers\`) needed an editorial restructure rather than a mechanical fix: Brown's real text is the hyphenated compound \`junior-senior high teachers\` (a school combining both grades), but this corpus had tokenized it as \`junior\` (sensed \`junior_high%1:14:00::\`) + \`senior_high\` (sensed \`senior_high%1:14:00::\`) -- apparently modeled on a different, superficially similar sentence elsewhere in the same document (\`a junior or senior_high_school teaching_certificate\`, where \`junior\` alone eliding "high school" after an explicit "or" is genuinely correct). Neither borrowed sense fits the real \`junior-senior\` compound, so retokenized to \`junior-senior\` + \`high\` and dropped both senses rather than force an inaccurate one, the same "drop when nothing fits" precedent as #24/#46.
Contributor guide
No contributing guide indexed for this repository
Research direction
Review src/semcor/underscore-hyphen-lexeme-fixes.yaml and the semcor-fix-underscore-hyphen-lexemes entry point, checking the named Brown source occurrences against their lemmas. Then inspect data/press_reportage/br-a02.yaml qwwa and verify that the 17 fixes and the junior-senior high retokenization match the stated corpus evidence, with no changes to the 74 valid candidates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100