bigscience-workshop / bigscience-workshop/data_tooling
Improve lexicon and ontology_manager.py
- Langage dominant
- HTML
- Étoiles
- 91
- Forks
- 47
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
Currently ontology_manager.py using a pattern match to find words potentially in a lexicon/ontology. You might want to have an exact dictionary to do a lexcion match, like "Barack Obama": "PUBLIC_FIGURE". Also, we need to incorporate better stop words and language specific stopwords to prevent the infering a word belongs in the lexicon when it's a stop words (like "maybe", "must", etc.)
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Piste de recherche
Start by reading ontology_manager.py and trace the current pattern-based lexicon or ontology matching. Determine where exact dictionary matching and stop-word filtering belong; done means matching supports entries such as "Barack Obama": "PUBLIC_FIGURE" and excludes general and language-specific stop words such as "maybe" and "must".
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- python
- Domaine
- data
- Type d'issue
- Fonctionnalité
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- À l'abandon
- Clarté
- Plutôt claire
- Accessibilité débutants
- 35/100