alteryx / alteryx/evalml

Spike: Potential improvements to LSA features in NaturalLanguageFeaturizer

Đang mở
#3,655 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
850
Fork
96
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

In the Featuretools LSA implementation, several cleaning steps are applied to the input data before computing the LSA feature values. The main steps are:
- removing punctuation
- converting all characters to lower case
- removing strings that contain numeric values such as `"user123"`
- removing stop words
- performing lemmatization

Many of these steps are not currently performed in EvalML, and the impact of performing these steps on model accuracy is unknown. We should study the impact of these steps to understand if adding them will improve accuracy. This could be done by studying the impact of these steps on several different problems in which natural language columns are present and the LSA features carry some importance.

The outcome of this spike should be a recommendation on whether to proceed with adding these steps to the EvalML implementation, and if so, follow up issues for implementing should be created.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.