HumanSignal / HumanSignal/label-studio

NER annotation text does not preserve spacing

Open
#3,078 0 comments 0 reactions 1 assignee Claimed by @makseq View on GitHub
export ner problem
Dominant language
TypeScript
Stars
28.3k
Forks
3.7k
Avg merge
14h
Merged PRs (30d)
15

Description

**Describe the bug**
When labeling NER, the annotation text may not match up with the original text at offsets if the text contains multiple whitespaces.

**To Reproduce**

1. Create task (note the double space)
```
echo "Text: Berlin, Germany." > task.txt
```
2. Open label-studio, create project with NER labeling config, import task

Labeling Config

```xml






```

3. Annotate the span `Berlin, German`
4. Observe annotation JSON

```json
{
"id": 1875,
"data": {
"text": "Text: Berlin, Germany."
},
"annotations": [
{
"id": 1036,
"result": [
{
"value": {
"start": 6,
"end": 22,
"text": "Berlin, Germany",
"labels": [
"LOC"
]
},
}
],
}
],
}
```

**Expected behavior**

I would expect `text[ann.start:ann.end] == ann.text`, but this is not the case (double whitespace is lost).

Overall, since the offsets are correct, this is not a big issue to me. Nevertheless, the behavior is not quite intuitive.

**Environment (please complete the following information):**
- OS: macOS 12.6 (21G115)
- Label Studio Version 1.6.0

Thanks for your help!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.