alteryx / alteryx/featuretools

Incorrect values for transform features that have the uses_calc_time property when using approximate and all aggregation features are approximated

未关闭
#661 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
bug
主要语言
Python
星标
7.7k
派生
915
PR 合并指标
30 天内没有已合并 PR

描述

In a special case that can occur while using the `approximate` option when calculating a feature matrix, transform features with primitives that have the `uses_calc_time` attribute (TimeSince is the only built-in primitive this applies to) are calculated with the `calc_time` set to the time when `calculate_feature_matrix` was called instead of the original cutoff time.

Here's an example:
```python
from datetime import datetime

import featuretools as ft
import pandas as pd
from featuretools.primitives import Count, Sum, TimeSince
from featuretools.tests.testing_utils import make_ecommerce_entityset

es = make_ecommerce_entityset()

# Feature to approximate
agg_feat = ft.Feature(es['log']['id'], parent_entity=es['sessions'], primitive=Count)
dfeat = ft.DirectFeature(agg_feat, es['log'])

# Cutoff times
times = [datetime(2011, 4, 9, 10, 31, 19), datetime(2011, 4, 9, 11, 0, 0)]
cutoff_time = pd.DataFrame({'time': times, 'instance_id': [0, 2]})

# Transform feature
trans_feat = ft.Feature(es['log']['datetime'], primitive=TimeSince)
```
Two features, and the direct feature can be approximated since it is a direct feature of an aggregation feature.

```python
# Unapproximated
feature_matrix = ft.calculate_feature_matrix([dfeat, trans_feat],
es,
cutoff_time=cutoff_time,
chunk_size="cutoff time")
```
Output:

| sessions.COUNT(log) | TIME_SINCE(datetime) |
| -- | -- |
|5 | 79.0 |
|5 | 1788.0 |

```python
# Approximated
feature_matrix = ft.calculate_feature_matrix([dfeat, trans_feat],
es,
approximate=ft.Timedelta(1, 'hour'),
cutoff_time=cutoff_time,
chunk_size="cutoff time")
```
Output:

| sessions.COUNT(log) | TIME_SINCE(datetime) |
| -- | -- |
| 0.0 | 2.605045e+08 |
| 5.0 | 2.605044e+08 |

The cause of this bug is a special code path taken when all aggregation features have been approximated that assumes it is safe to calculate all transform features using the current time as the cutoff time. Transform primitives with `uses_calc_time` still need to be calculated using their original cutoff time.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。