stdlib-js / stdlib-js/stdlib

[RFC]: Improvements to @stdlib/nlp-expand-contractions

Open
#496 8 comments 0 reactions 0 assignees View on GitHub
Feature RFC
Dominant language
JavaScript
Stars
6k
Forks
1.3k
Avg merge
1d 3h
Merged PRs (30d)
611

Description

### Description

We're writing as we found your library to be the most tested and fastest for expanding contractions. For context, we're working on https://spamscanner.net and expanding contractions before passing to tokenizers for spam classification.

To clarify, this is with regards to the generated codebase from the source at .

We noticed that your library is missing quite a few contractions in English, and could also benefit from contractions from other languages too (perhaps with an option).

While we can open a PR, we wanted to check to see what your thoughts were on this and how you might want the PR to look like (integration wise; e.g. new options?).

Here is our current compiled list of research and findings:

- List of contractions research
- https://en.wikipedia.org/wiki/Wikipedia:List_of_English_contractions
- https://en.wiktionary.org/wiki/Category:English_double_contractions
- https://gist.github.com/loretoparisi/c221a9c55fb71a23ff4e7bba3b794425?permalink_comment_id=4198425
- https://www.enchantedlearning.com/grammar/contractions/
- https://github.com/NaturalNode/natural/issues/533
- https://github.com/anton-bot/expand-contractions/pull/1
- https://github.com/kootenpv/contractions/blob/master/contractions/data/leftovers_dict.json
- https://github.com/kootenpv/contractions/blob/master/contractions/data/slang_dict.json
- https://github.com/stdlib-js/nlp-expand-contractions/blob/main/lib/contractions.json
- https://github.com/stdlib-js/nlp-expand-contractions/blob/main/lib/expand_contractions.js
- https://github.com/textlint-rule/textlint-rule-preset-google/blob/master/packages/textlint-rule-google-contractions/src/textlint-rule-google-contractions.js#L67-L87
- https://web.library.yale.edu/cataloging/months
- https://www.wikidata.org/w/index.php?search=%2B%22Category%3A%22+%2B%22contractions%22&title=Special:Search&profile=advanced&fulltext=1&ns0=1&ns120=1
- There are so many other lists that we can scrape:
- For example, French contractions: https://en.wiktionary.org/wiki/Category:French_contractions
- Here are other contractions we think should be included (that were not found elsewhere)
- 'twas -> it was
- 'tisn't -> it is not
- ma'am -> madam
- mightn't've -> might not have
- mustn't've -> must not have
- ne'er-do-well -> never do well
- o' -> of
- o'clock -> of the clock
- she'd've -> she would have
- shouldn't've -> should not have
- wouldn't've -> would not have

### Related Issues

_No response_

### Questions

_No response_

### Other

_No response_

### Checklist

- [X] I have read and understood the [Code of Conduct](https://github.com/stdlib-js/stdlib/blob/develop/CODE_OF_CONDUCT.md).
- [X] Searched for existing issues and pull requests.
- [X] The issue name begins with `RFC:`.

Contributor guide

Open the contributing guide

Research direction

Start with lib/contractions.json and lib/expand_contractions.js in the generated package, then review the linked research and clarify the supported languages and integration options with maintainers. Done means an agreed design and an accepted update covering the selected contractions and expansion behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.