WikipediaTokenizer incorrectly splits certain syntax into multiple tokens [LUCENE-1141]
Open
legacy-jira-priority:Minor
module:analysis
type:bug
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
WikipediaTokenizer incorrectly splits tokens that have italics/bold inside the token, for instance '''F'''oo is a bold Foo, not F, oo
---
Migrated from [LUCENE-1141](https://issues.apache.org/jira/browse/LUCENE-1141) by Grant Ingersoll (@gsingers), updated May 16 2011
Attachments: [LUCENE-1141-test.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-1141/LUCENE-1141-test.patch)
Contributor guide
Assessment
This issue has not been assessed yet.