alteryx / alteryx/evalml

Implement consistent cleaning for NatualLanguageFeaturizer

Aperta
#3,654 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
850
Fork
96
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

In the `NaturalLanguageFeaturizer` component, cleaning of the data happens in the `_make_entityset` method. The LSA primitive is fit before the data cleaning happens, which I believe means that the custom LSA corpus is created using data that had not yet been cleaned.

However, when transform is called on the LSA component, this happens after `_make_entityset` has been called, and it appears that the cleaning of the dataframe happens in place. If I have followed things correctly, this would mean that LSA is fit using unclean data, but transform uses cleaned data to calculate the feature values.

This should be investigated further and if this is found to be an issue, consistent cleaning should be implemented to make sure that both fit and transform use the same clean data for computing LSA feature values.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.