aboutcode-org / aboutcode-org/scancode-toolkit
[GSoC] Build NER training dataset from annotated rules
Abierto
GSoC
- Lenguaje dominante
- Python
- Estrellas
- 2.6k
- Forks
- 791
- Merge medio
- 1 d 12 h
- PR fusionados (30 d)
- 5
Descripción
Tasks:
- Parse .RULE files and extract extract required phrase markers
- Propagate annotations (with CLI commands from #3924 ) to unannotated rules (with matching text) to expand the dataset
- Improve composite rules (AND/OR expressions where some aren't marked)
- Convert to BIOES labels
- Split into train/val/test by rule family
- Validate dataset quality
Timeline: Weeks 1-2 (current)
Deliverable: BIOES dataset ready for training (train/val/test splits)
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.