aboutcode-org / aboutcode-org/scancode-toolkit

[GSoC] Build NER training dataset from annotated rules

Ouverte
#5,077 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
GSoC
Langage dominant
Python
Étoiles
2.6k
Forks
791
Merge moyen
1 j 12 h
PR mergées (30 j)
5

Description

Tasks:
- Parse .RULE files and extract extract required phrase markers
- Propagate annotations (with CLI commands from #3924 ) to unannotated rules (with matching text) to expand the dataset
- Improve composite rules (AND/OR expressions where some aren't marked)
- Convert to BIOES labels
- Split into train/val/test by rule family
- Validate dataset quality

Timeline: Weeks 1-2 (current)

Deliverable: BIOES dataset ready for training (train/val/test splits)

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.