alteryx / alteryx/featuretools

Improve EntitySet equality Checks

未关闭
#1,328 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
tech debt
主要语言
Python
星标
7.7k
派生
915
PR 合并指标
30 天内没有已合并 PR

描述

### Improve EntitySet equality Checks

#### Bug/Feature Request Description

The current Featuretools code uses a custom `_dataframes_equal` function to check for equality between dataframes, rather than testing equality via pandas (https://github.com/alteryx/featuretools/blob/46cc496ab21909d019278ab8f29f7686cbc0c549/featuretools/utils/wrangle.py#L115).

The custom function **might** have performance advantages, but this comes at the expense of more complicated code that need to be maintained. Additionally, the current code will determine that two dataframes with a different number of rows are equal (see code below), which also could result in two EntitySets being treated as equal even when their dataframes are different.

We should review the usage of this custom function and possibly replace this with built-in pandas methods instead, but performance benchmarking should be done to make sure we don't take a large performance hit when making this update.

```python
import featuretools as ft
from featuretools.utils.wrangle import _dataframes_equal
df_list = ft.demo.load_mock_customer()
df = df_list['transactions']
_dataframes_equal(df, df.head(1))
>>> True
df.equals(df.head(1))
>>> False
es1 = ft.EntitySet(df)
es2 = ft.EntitySet(df.head(1))
es1.__eq__(es2, deep=True)
>>> True
```

#### Expected Output
I expect that two entitysets that contain dataframes that are not equal, should not be treated as equal during a `deep` equality check.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。