alteryx / alteryx/featuretools
Incorrect values for transform features that have the uses_calc_time property when using approximate and all aggregation features are approximated
- 主要语言
- Python
- 星标
- 7.7k
- 派生
- 915
- PR 合并指标
- 30 天内没有已合并 PR
描述
In a special case that can occur while using the `approximate` option when calculating a feature matrix, transform features with primitives that have the `uses_calc_time` attribute (TimeSince is the only built-in primitive this applies to) are calculated with the `calc_time` set to the time when `calculate_feature_matrix` was called instead of the original cutoff time.
Here's an example:
```python
from datetime import datetime
import featuretools as ft
import pandas as pd
from featuretools.primitives import Count, Sum, TimeSince
from featuretools.tests.testing_utils import make_ecommerce_entityset
es = make_ecommerce_entityset()
# Feature to approximate
agg_feat = ft.Feature(es['log']['id'], parent_entity=es['sessions'], primitive=Count)
dfeat = ft.DirectFeature(agg_feat, es['log'])
# Cutoff times
times = [datetime(2011, 4, 9, 10, 31, 19), datetime(2011, 4, 9, 11, 0, 0)]
cutoff_time = pd.DataFrame({'time': times, 'instance_id': [0, 2]})
# Transform feature
trans_feat = ft.Feature(es['log']['datetime'], primitive=TimeSince)
```
Two features, and the direct feature can be approximated since it is a direct feature of an aggregation feature.
```python
# Unapproximated
feature_matrix = ft.calculate_feature_matrix([dfeat, trans_feat],
es,
cutoff_time=cutoff_time,
chunk_size="cutoff time")
```
Output:
| sessions.COUNT(log) | TIME_SINCE(datetime) |
| -- | -- |
|5 | 79.0 |
|5 | 1788.0 |
```python
# Approximated
feature_matrix = ft.calculate_feature_matrix([dfeat, trans_feat],
es,
approximate=ft.Timedelta(1, 'hour'),
cutoff_time=cutoff_time,
chunk_size="cutoff time")
```
Output:
| sessions.COUNT(log) | TIME_SINCE(datetime) |
| -- | -- |
| 0.0 | 2.605045e+08 |
| 5.0 | 2.605044e+08 |
The cause of this bug is a special code path taken when all aggregation features have been approximated that assumes it is safe to calculate all transform features using the current time as the cutoff time. Transform primitives with `uses_calc_time` still need to be calculated using their original cutoff time.
贡献指南
评估
这个 Issue 还没有评估数据。