Spike: Potential improvements to LSA features in NaturalLanguageFeaturizer
- Lingua principale
- Python
- Stelle
- 850
- Fork
- 96
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
In the Featuretools LSA implementation, several cleaning steps are applied to the input data before computing the LSA feature values. The main steps are:
- removing punctuation
- converting all characters to lower case
- removing strings that contain numeric values such as `"user123"`
- removing stop words
- performing lemmatization
Many of these steps are not currently performed in EvalML, and the impact of performing these steps on model accuracy is unknown. We should study the impact of these steps to understand if adding them will improve accuracy. This could be done by studying the impact of these steps on several different problems in which natural language columns are present and the LSA features carry some importance.
The outcome of this spike should be a recommendation on whether to proceed with adding these steps to the EvalML implementation, and if so, follow up issues for implementing should be created.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.