explosion / explosion/spaCy

Add checks (and converters?) for documents with multiple sentences in debug-data

Open
#4,409 1 comment 2 reactions 1 assignee Claimed by @adrianeboyd View on GitHub
enhancement feat / cli
Dominant language
Python
Stars
33.9k
Forks
4.7k
Avg merge
3m
Merged PRs (30d)
1

Description

## Feature description

The parser section of `spacy debug-data` should show a warning when there are no/few documents with multiple sentences in the training data.

Potentially add a simple converter to `spacy convert` to group sentences, similar to `-n` with the IOB converters. A bit of variety in document lengths is probably a good idea here, too, rather than just `-n N`, but I don't know if it makes that much difference in the model performance.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.