explosion / explosion/spaCy

"invalid whitespace entity spans" error while validation training and test data for NER

Open
#13,689 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
33.9k
Forks
4.7k
Avg merge
3m
Merged PRs (30d)
1

Description

## How to reproduce the behaviour

I have use the following piece of code to convert json to spacy while validationg using spacy --debug i get whitespace error:

![image](https://github.com/user-attachments/assets/b2176cf3-35d1-4d0d-81d0-94c284bc219d)

please help me how to resolve this

for text, annot in tqdm(TRAIN_DATA['annotations']):
doc = nlp.make_doc(text)
ents = []
for start, end, label in annot["entities"]:
span = doc.char_span(start, end, label=label, alignment_mode="contract")
if span is None:
print("Skipping entity")
else:
ents.append(span)
doc.ents = ents
db.add(doc)
db.to_disk("train_data.spacy")

## Info about spaCy

- **spaCy version:** 3.7.5
- **Platform:** Linux-6.1.85+-x86_64-with-glibc2.35
- **Python version:** 3.10.12
- **Pipelines:** en_core_web_lg (3.7.1), en_core_web_sm (3.7.1)

* Operating System:
* Python Version Used:
* spaCy Version Used:
* Environment Information:

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.