alteryx / alteryx/evalml

Spike: Potential improvements to LSA features in NaturalLanguageFeaturizer

Aberta
#3,655 0 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
850
Forks
96
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

In the Featuretools LSA implementation, several cleaning steps are applied to the input data before computing the LSA feature values. The main steps are:
- removing punctuation
- converting all characters to lower case
- removing strings that contain numeric values such as `"user123"`
- removing stop words
- performing lemmatization

Many of these steps are not currently performed in EvalML, and the impact of performing these steps on model accuracy is unknown. We should study the impact of these steps to understand if adding them will improve accuracy. This could be done by studying the impact of these steps on several different problems in which natural language columns are present and the LSA features carry some importance.

The outcome of this spike should be a recommendation on whether to proceed with adding these steps to the EvalML implementation, and if so, follow up issues for implementing should be created.

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.