aboutcode-org / aboutcode-org/scancode-toolkit
[GSoC] Build NER training dataset from annotated rules
Aberta
GSoC
- Linguagem predominante
- Python
- Estrelas
- 2.6k
- Forks
- 791
- Merge médio
- 1d 12h
- PRs com merge (30d)
- 5
Descrição
Tasks:
- Parse .RULE files and extract extract required phrase markers
- Propagate annotations (with CLI commands from #3924 ) to unannotated rules (with matching text) to expand the dataset
- Improve composite rules (AND/OR expressions where some aren't marked)
- Convert to BIOES labels
- Split into train/val/test by rule family
- Validate dataset quality
Timeline: Weeks 1-2 (current)
Deliverable: BIOES dataset ready for training (train/val/test splits)
Guia de contribuição
Avaliação
Esta issue ainda não foi avaliada.