stdlib-js / stdlib-js/stdlib

[RFC]: Improvements to @stdlib/nlp-expand-contractions

Abierto
#496 8 comentarios 0 reacciones 0 asignados Ver en GitHub
Feature RFC
Lenguaje dominante
JavaScript
Estrellas
6k
Forks
1.3k
Merge medio
1 d 3 h
PR fusionados (30 d)
611

Descripción

### Description

We're writing as we found your library to be the most tested and fastest for expanding contractions. For context, we're working on https://spamscanner.net and expanding contractions before passing to tokenizers for spam classification.

To clarify, this is with regards to the generated codebase from the source at .

We noticed that your library is missing quite a few contractions in English, and could also benefit from contractions from other languages too (perhaps with an option).

While we can open a PR, we wanted to check to see what your thoughts were on this and how you might want the PR to look like (integration wise; e.g. new options?).

Here is our current compiled list of research and findings:

- List of contractions research
- https://en.wikipedia.org/wiki/Wikipedia:List_of_English_contractions
- https://en.wiktionary.org/wiki/Category:English_double_contractions
- https://gist.github.com/loretoparisi/c221a9c55fb71a23ff4e7bba3b794425?permalink_comment_id=4198425
- https://www.enchantedlearning.com/grammar/contractions/
- https://github.com/NaturalNode/natural/issues/533
- https://github.com/anton-bot/expand-contractions/pull/1
- https://github.com/kootenpv/contractions/blob/master/contractions/data/leftovers_dict.json
- https://github.com/kootenpv/contractions/blob/master/contractions/data/slang_dict.json
- https://github.com/stdlib-js/nlp-expand-contractions/blob/main/lib/contractions.json
- https://github.com/stdlib-js/nlp-expand-contractions/blob/main/lib/expand_contractions.js
- https://github.com/textlint-rule/textlint-rule-preset-google/blob/master/packages/textlint-rule-google-contractions/src/textlint-rule-google-contractions.js#L67-L87
- https://web.library.yale.edu/cataloging/months
- https://www.wikidata.org/w/index.php?search=%2B%22Category%3A%22+%2B%22contractions%22&title=Special:Search&profile=advanced&fulltext=1&ns0=1&ns120=1
- There are so many other lists that we can scrape:
- For example, French contractions: https://en.wiktionary.org/wiki/Category:French_contractions
- Here are other contractions we think should be included (that were not found elsewhere)
- 'twas -> it was
- 'tisn't -> it is not
- ma'am -> madam
- mightn't've -> might not have
- mustn't've -> must not have
- ne'er-do-well -> never do well
- o' -> of
- o'clock -> of the clock
- she'd've -> she would have
- shouldn't've -> should not have
- wouldn't've -> would not have

### Related Issues

_No response_

### Questions

_No response_

### Other

_No response_

### Checklist

- [X] I have read and understood the [Code of Conduct](https://github.com/stdlib-js/stdlib/blob/develop/CODE_OF_CONDUCT.md).
- [X] Searched for existing issues and pull requests.
- [X] The issue name begins with `RFC:`.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.