bigscience-workshop / bigscience-workshop/data_tooling

Improve lexicon and ontology_manager.py

Open
#274 0 comments 0 reactions 0 assignees View on GitHub
PII
Dominant language
HTML
Stars
91
Forks
47
PR merge metrics
No merged PRs in 30d

Description

Currently ontology_manager.py using a pattern match to find words potentially in a lexicon/ontology. You might want to have an exact dictionary to do a lexcion match, like "Barack Obama": "PUBLIC_FIGURE". Also, we need to incorporate better stop words and language specific stopwords to prevent the infering a word belongs in the lexicon when it's a stop words (like "maybe", "must", etc.)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.