apache / apache/lucene

HTMLStripCharFilter += HTML5 [LUCENE-5763]

Open
#6,825 4 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor module:analysis type:task
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

HTMLStripCharFilter knows some specific things about HTML4 (like named character entities, which are converted to the corresponding characters), but not about HTML5.

HTML5 has way more named character entities: 2,231 vs 259 by my count.

There's probably other stuff to do, e.g. there are new tags.

---
Migrated from [LUCENE-5763](https://issues.apache.org/jira/browse/LUCENE-5763) by Steven Rowe (@sarowe), updated Jun 19 2014

Contributor guide

Open the contributing guide

Research direction

Start by reading HTMLStripCharFilter and identify which HTML4 entities and tags it currently handles. Compare that behavior with the HTML5 requirements described in the issue, then define the supported scope and validation needed before implementation; done means the agreed HTML5 behavior is covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.