bigscience-workshop / bigscience-workshop/data_tooling
Improve lexicon and ontology_manager.py
Open
PII
- Dominant language
- HTML
- Stars
- 91
- Forks
- 47
- PR merge metrics
- No merged PRs in 30d
Description
Currently ontology_manager.py using a pattern match to find words potentially in a lexicon/ontology. You might want to have an exact dictionary to do a lexcion match, like "Barack Obama": "PUBLIC_FIGURE". Also, we need to incorporate better stop words and language specific stopwords to prevent the infering a word belongs in the lexicon when it's a stop words (like "maybe", "must", etc.)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.