alteryx / alteryx/evalml

Implement consistent cleaning for NatualLanguageFeaturizer

Offen
#3,654 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
850
Forks
96
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

In the `NaturalLanguageFeaturizer` component, cleaning of the data happens in the `_make_entityset` method. The LSA primitive is fit before the data cleaning happens, which I believe means that the custom LSA corpus is created using data that had not yet been cleaned.

However, when transform is called on the LSA component, this happens after `_make_entityset` has been called, and it appears that the cleaning of the dataframe happens in place. If I have followed things correctly, this would mean that LSA is fit using unclean data, but transform uses cleaned data to calculate the feature values.

This should be investigated further and if this is found to be an issue, consistent cleaning should be implemented to make sure that both fit and transform use the same clean data for computing LSA feature values.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.