bigscience-workshop / bigscience-workshop/data_tooling
Improve lexicon and ontology_manager.py
- 主要语言
- HTML
- 星标
- 91
- 派生
- 47
- PR 合并指标
- 30 天内没有已合并 PR
描述
Currently ontology_manager.py using a pattern match to find words potentially in a lexicon/ontology. You might want to have an exact dictionary to do a lexcion match, like "Barack Obama": "PUBLIC_FIGURE". Also, we need to incorporate better stop words and language specific stopwords to prevent the infering a word belongs in the lexicon when it's a stop words (like "maybe", "must", etc.)
贡献指南
这个仓库没有索引到贡献指南
调研方向
Start by reading ontology_manager.py and trace the current pattern-based lexicon or ontology matching. Determine where exact dictionary matching and stop-word filtering belong; done means matching supports entries such as "Barack Obama": "PUBLIC_FIGURE" and excludes general and language-specific stop words such as "maybe" and "must".
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- data
- Issue 类型
- 功能
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100