Use SQLite ICU extension to enable case-insensitive searches for non-Latin characters
- Dominant language
- Rust
- Stars
- 30.5k
- Forks
- 3.2k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 72
Description
This is an enhancement request for Anki to consider using the SQLite [ICU extension](https://sqlite.org/src/dir/ext/icu). As I understand it, doing so will result in case-insensitive simple search functionality for all characters. Currently, simple search is case-insensitive only for Latin characters.
It may be that using the ICU extension will result in slower searches. If the increase in search time is too great, it may be that this enhancement request should be rejected and closed.
---
https://www.sqlite.org/lang_expr.html#like
> SQLite only understands upper/lower case for ASCII characters by default. The LIKE operator is case sensitive by default for unicode characters that are beyond the ASCII range. [...] The ICU extension to SQLite includes an enhanced version of the LIKE operator that does case folding across all unicode characters.
https://sqlite.org/src/dir/ext/icu
> [...] the built-in SQLite LIKE
> operator understands case equivalence for the 26 letters of the English
> language alphabet. The implementation of LIKE included in this
> extension uses the ICU function u_foldCase() to provide case
> independent comparisons for the full range of unicode characters.
https://docs.ankiweb.net/searching.html#simple-searches
> Standard searches are case insensitive for Latin characters - a-z will match A-Z, and vice versa. Other characters such as Cyrillic are case sensitive in a standard search, but can be made case insensitive by searching on a word boundary or regular expression (w:, re:).
Although the current Anki search behavior is documented, it resulted in a user believing that they were seeing a bug and posting on the forum:
https://forums.ankiweb.net/t/the-problem-with-the-register-upper-lowercase-searching/66105
Contributor guide
Research direction
Start by reading the SQLite ICU extension documentation and Anki's simple-search behavior described in docs. The work would need to assess integrating Unicode case folding, measure the effect on search performance, and establish whether non-Latin case-insensitive searches improve enough to justify the change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sqlite
- Domain
- database
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100