globalwordnet / globalwordnet/semcor

17 more underscore-joined hyphenated lexemes missed by #43, plus one editorial restructure (junior-senior high)

Open
#64 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
Avg merge
2d 22h
Merged PRs (30d)
24

Description

Re-scanning for #43's own pattern (a token's surface uses \`_\` where its own \`lemmas\` entry already has \`-\` at the same position, e.g. \`ex_gambler\` with lemma \`ex-gambler\`) finds 17 more instances #43's original scan missed, each individually confirmed against \`src/semcor/brown-nolines.txt\`. Added to \`src/semcor/underscore-hyphen-lexeme-fixes.yaml\` (54 -> 71) and fixed via the existing \`semcor-fix-underscore-hyphen-lexemes\`, unchanged.

Of 91 raw candidates (same lemma-vs-surface signal), 74 were correctly left alone: the lemma's hyphen is just WordNet's own dictionary-citation-form convention for the multiword entry (\`such-and-such\`, \`upside-down\`, \`half hour\` in this specific instance, ...) and Brown's real text at that exact position genuinely uses a plain space, not a hyphen -- confirmed per-instance, since Brown is genuinely inconsistent between space and hyphen for some of these across different occurrences (e.g. \`half hour\`/\`half-hour\` both appear elsewhere).

Separately, \`data/press_reportage/br-a02.yaml\`'s \`qwwa\` (\`junior senior_high teachers\`) needed an editorial restructure rather than a mechanical fix: Brown's real text is the hyphenated compound \`junior-senior high teachers\` (a school combining both grades), but this corpus had tokenized it as \`junior\` (sensed \`junior_high%1:14:00::\`) + \`senior_high\` (sensed \`senior_high%1:14:00::\`) -- apparently modeled on a different, superficially similar sentence elsewhere in the same document (\`a junior or senior_high_school teaching_certificate\`, where \`junior\` alone eliding "high school" after an explicit "or" is genuinely correct). Neither borrowed sense fits the real \`junior-senior\` compound, so retokenized to \`junior-senior\` + \`high\` and dropped both senses rather than force an inaccurate one, the same "drop when nothing fits" precedent as #24/#46.

Contributor guide

No contributing guide indexed for this repository

Research direction

Review src/semcor/underscore-hyphen-lexeme-fixes.yaml and the semcor-fix-underscore-hyphen-lexemes entry point, checking the named Brown source occurrences against their lemmas. Then inspect data/press_reportage/br-a02.yaml qwwa and verify that the 17 fixes and the junior-senior high retokenization match the stated corpus evidence, with no changes to the 74 valid candidates.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.