alteryx / alteryx/evalml

Spike: Potential improvements to LSA features in NaturalLanguageFeaturizer

Offen
#3,655 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
850
Forks
96
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

In the Featuretools LSA implementation, several cleaning steps are applied to the input data before computing the LSA feature values. The main steps are:
- removing punctuation
- converting all characters to lower case
- removing strings that contain numeric values such as `"user123"`
- removing stop words
- performing lemmatization

Many of these steps are not currently performed in EvalML, and the impact of performing these steps on model accuracy is unknown. We should study the impact of these steps to understand if adding them will improve accuracy. This could be done by studying the impact of these steps on several different problems in which natural language columns are present and the LSA features carry some importance.

The outcome of this spike should be a recommendation on whether to proceed with adding these steps to the EvalML implementation, and if so, follow up issues for implementing should be created.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.