corejavascript / corejavascript/typeahead.js
include uniqueness filter in noramlizeTokens
Open
- Dominant language
- JavaScript
- Stars
- 968
- Forks
- 231
- PR merge metrics
- No merged PRs in 30d
Description
https://github.com/corejavascript/typeahead.js/blob/662ee462829fb977bdad59a5707db0db65335f9c/src/bloodhound/search_index.js#L136
I was looking at issue #183 regarding starts with logic. This seems to be a result of how the Ngram tokenizer works. I was playing around with implementing a tokenizer that would output interword tokens. It would generate a lot more tokens. I noticed normalizeTokens does not return unique list tokens. It seems like getIntersection efficiency would benefit from being fed a unique list? Would there be any downsides to this?
Contributor guide
Assessment
This issue has not been assessed yet.