alteryx / alteryx/featuretools

EPIC: Refactor `DeepFeatureSynthesis._build_features`

未关闭
#2,106 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
enhancement refactor tech debt
主要语言
Python
星标
7.7k
派生
915
PR 合并指标
30 天内没有已合并 PR

描述

`DeepFeatureSynthesis._build_features` is in need of a refactor to improve speed, maintainability, and scalability.

There are many optimizations that can be made underneath this function to improve performance while maintaining API signature. As a rough benchmark, the `get_valid_primitives` function takes 2 hours to run on the retail entityset to produce a little over 5 million feature defintions. This can be optimized to be much take a much shorter time.

## Functions should be more granular and testable:

For example one of the most granular functions should take a datastructure which is a hashmap of features, keyed by their ColumnSchema as a single argument, and another argument which is an inputset (eg. Numeric, Boolean), and return a list of lists of all feature combinations that match this inputtype signature. This function should be pure, which would improve maintainability by being very readable and testable.

## Optimizations:

### Caching
Using the example above, this function could be wrapped with an LRU Cache decorator that would allow primitives that have input signatures matching other primitives to return immediately. Memory issues should be of little concern since these calculations can be perfomed using very datastructures containing logical types only and no data, but this should be measured and tested.

### Data Structures
Features and primitives should be hashed by their associated logical types for faster lookup.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。