CentreForDigitalHumanities / CentreForDigitalHumanities/TextMiNER
Adjust tokenizer
Open
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Flair's tokenizer seems to remove newline characters. This is not always the desired behaviour: documents derived from OCR of scans may have meaningful linebreaks.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.