globalwordnet / globalwordnet/semcor
Strip **h/**H typesetting marker from the reference comparison
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 24
Description
\`**h\`/\`**H\` has no corresponding character anywhere in this corpus's own data -- confirmed against several instances in \`data/press_reportage/br-a12.yaml\`: \`"40 per cent **h their total passing yardage"\` and \`"86 tries **h. Tailback"\` in \`brown-nolines.txt\` both have nothing at all where \`**h\` sits in this corpus's own \`text\` (\`"per cent their"\`, \`"tries. Tailback"\`). Unlike \`**f\`, which is tangled up with the already-tracked formula-placeholder gap (#16/#34), \`**h\` is clean reference-side noise.
Dropped in \`decode_reference_text\`, the same as the paragraph-break tokens, swallowing an optional leading space so two real words end up with exactly the single space that was already between them, while a trailing character with no space of its own (\`tries **h.\` -> \`tries.\`) is left in place.
Divergent word-line count drops by 508 with no \`data/*.yaml\` changes.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in decode_reference_text and inspect how paragraph-break tokens are handled, using the examples in data/press_reportage/br-a12.yaml and brown-nolines.txt as reference cases. Strip the optional leading space with **h/**H while preserving the character after a token with no space of its own; done is a 508-line drop in divergent word-line counts with no data/*.yaml changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100