Spike: Potential improvements to LSA features in NaturalLanguageFeaturizer
- Linguagem predominante
- Python
- Estrelas
- 850
- Forks
- 96
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
In the Featuretools LSA implementation, several cleaning steps are applied to the input data before computing the LSA feature values. The main steps are:
- removing punctuation
- converting all characters to lower case
- removing strings that contain numeric values such as `"user123"`
- removing stop words
- performing lemmatization
Many of these steps are not currently performed in EvalML, and the impact of performing these steps on model accuracy is unknown. We should study the impact of these steps to understand if adding them will improve accuracy. This could be done by studying the impact of these steps on several different problems in which natural language columns are present and the LSA features carry some importance.
The outcome of this spike should be a recommendation on whether to proceed with adding these steps to the EvalML implementation, and if so, follow up issues for implementing should be created.
Guia de contribuição
Avaliação
Esta issue ainda não foi avaliada.