DataTables / DataTables/Plugins

diacritics-neutralise has trouble when characters with modifiers are used

Open
#401 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
1.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

In Unicode you can make a character like "ý" as either a single standalone character U+00FD or a 'y' followed by an accent, U+0079U+0301.

diacritics-neutralise doesn't really even try to mediate between these.

A relatively simple possible solution is to use the normalize() method on javascript strings.

function removeDiacritics(s) {
  s = s.normalize();
  for (var i = 0, l = diacriticsMap.length; i < l; i++) {
    s = s.replace(diacriticsMap[i].letters, diacriticsMap[i].base);
  }
  return s;
}

I've done this to my local copy of the code. I don't know yet if there will be any ill effects, but I suspect not, since both the search and target strings will be normalized.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the diacritics-neutralise implementation and its removeDiacritics function. Check how JavaScript string normalization interacts with the existing diacritics map, then verify behavior for both composed and decomposed forms of the accented character. Done means equivalent Unicode forms are handled consistently without breaking existing replacements.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
search
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.