[RFC]: Improvements to @stdlib/nlp-expand-contractions
- Dominant language
- JavaScript
- Stars
- 6k
- Forks
- 1.3k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 611
Description
### Description
We're writing as we found your library to be the most tested and fastest for expanding contractions. For context, we're working on https://spamscanner.net and expanding contractions before passing to tokenizers for spam classification.
To clarify, this is with regards to the generated codebase from the source at .
We noticed that your library is missing quite a few contractions in English, and could also benefit from contractions from other languages too (perhaps with an option).
While we can open a PR, we wanted to check to see what your thoughts were on this and how you might want the PR to look like (integration wise; e.g. new options?).
Here is our current compiled list of research and findings:
- List of contractions research
- https://en.wikipedia.org/wiki/Wikipedia:List_of_English_contractions
- https://en.wiktionary.org/wiki/Category:English_double_contractions
- https://gist.github.com/loretoparisi/c221a9c55fb71a23ff4e7bba3b794425?permalink_comment_id=4198425
- https://www.enchantedlearning.com/grammar/contractions/
- https://github.com/NaturalNode/natural/issues/533
- https://github.com/anton-bot/expand-contractions/pull/1
- https://github.com/kootenpv/contractions/blob/master/contractions/data/leftovers_dict.json
- https://github.com/kootenpv/contractions/blob/master/contractions/data/slang_dict.json
- https://github.com/stdlib-js/nlp-expand-contractions/blob/main/lib/contractions.json
- https://github.com/stdlib-js/nlp-expand-contractions/blob/main/lib/expand_contractions.js
- https://github.com/textlint-rule/textlint-rule-preset-google/blob/master/packages/textlint-rule-google-contractions/src/textlint-rule-google-contractions.js#L67-L87
- https://web.library.yale.edu/cataloging/months
- https://www.wikidata.org/w/index.php?search=%2B%22Category%3A%22+%2B%22contractions%22&title=Special:Search&profile=advanced&fulltext=1&ns0=1&ns120=1
- There are so many other lists that we can scrape:
- For example, French contractions: https://en.wiktionary.org/wiki/Category:French_contractions
- Here are other contractions we think should be included (that were not found elsewhere)
- 'twas -> it was
- 'tisn't -> it is not
- ma'am -> madam
- mightn't've -> might not have
- mustn't've -> must not have
- ne'er-do-well -> never do well
- o' -> of
- o'clock -> of the clock
- she'd've -> she would have
- shouldn't've -> should not have
- wouldn't've -> would not have
### Related Issues
_No response_
### Questions
_No response_
### Other
_No response_
### Checklist
- [X] I have read and understood the [Code of Conduct](https://github.com/stdlib-js/stdlib/blob/develop/CODE_OF_CONDUCT.md).
- [X] Searched for existing issues and pull requests.
- [X] The issue name begins with `RFC:`.
Contributor guide
Research direction
Start with lib/contractions.json and lib/expand_contractions.js in the generated package, then review the linked research and clarify the supported languages and integration options with maintainers. Done means an agreed design and an accepted update covering the selected contractions and expansion behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100