Lowercase lemmatization in pipe, when tok2vec disabled
- Dominant language
- Python
- Stars
- 33.9k
- Forks
- 4.7k
- Avg merge
- 3m
- Merged PRs (30d)
- 1
Description
## Introduction
Hi! I am using spaCy lemmatizer for some tasks. I saw that when using a pipe to process the data faster, I'm getting different results with tok2vec disabled or enabled. Maintaining case-sensitivity is critical for me. Is the below behavior expected?
## How to reproduce the behaviour
### Case1
```python
import spacy
nlp = spacy.load("en_core_web_sm")
for doc in nlp.pipe(["Hello! My name is Marcin.", "I have a SFTP server running in my HomeLab"], batch_size=100, n_process=1, disable=["ner", "tok2vec"]):
for token in doc:
print(str(token), token.lemma_)
print("")
```
Output:
```
Hello hello
! !
My my
name name
is is
Marcin marcin
. .
I i
have have
a a
SFTP sftp
server server
running running
in in
my my
HomeLab homelab
```
### Case2
```python
import spacy
nlp = spacy.load("en_core_web_sm")
for doc in nlp.pipe(["Hello! My name is Marcin.", "I have a SFTP server running in my HomeLab"], batch_size=100, n_process=1, disable=["ner"]):
for token in doc:
print(str(token), token.lemma_)
print("")
```
Output:
```
Hello hello
! !
My my
name name
is be
Marcin Marcin
. .
I I
have have
a a
SFTP sftp
server server
running run
in in
my my
HomeLab HomeLab
```
## Info about spaCy
- **spaCy version:** 3.7.2
- **Platform:** Windows-10-10.0.19045-SP0
- **Python version:** 3.10.13
- **Pipelines:** en_core_web_sm (3.7.1), en_core_web_trf (3.7.3), es_core_news_lg (3.7.0), es_core_news_sm (3.7.0), pl_core_news_lg (3.7.0), pl_core_news_sm (3.7.0)
Contributor guide
Assessment
This issue has not been assessed yet.