alteryx / alteryx/woodwork

Improve numeric inference capabilities Double/Integer/WholeNumber

Đang mở
#140 3 bình luận 0 reaction 0 người được giao Xem trên GitHub
enhancement
Ngôn ngữ chính
Python
Star
155
Fork
24
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

In the current approach to logical type inference the main differentiator between a `Double` and an `Integer` or `WholeNumber` logical type is the underlying pandas `dtype`. So, in the following examples, these series would be inferred to have a logical type of `Double` when a `WholeNumber` or `Integer` type would be more appropriate.

```python
from woodwork.data_column import infer_logical_type

# inferred as Double because dtype is float64, but could be WholeNumber with Int64 dtype
pd.Series([1.0, 2.0, -3.0])
>>> infer_logical_type(pd.Series([1.0, np.nan, 3.0]))
Double

# inferred as Double because dtype is float64, but could be Integer with Int64 dtype
>>> infer_logical_type(pd.Series([1.0, 2.0, -3.0]))
Double
```

We should improve the inference for series that contain `NaN` values or for series that can be represented as `Integer` or `WholeNumber` types without loss of information. One way to check for this loss of information would be to cast columns inferred as `Double` to integer and then determine if the values are equal to the original float values or not. Would need to drop `NaN` values first to perform this comparison.

```python
>>> all(pd.Series([1.0, 2.0]).astype('int') == pd.Series([1.0, 2.0]))
True
>>> all(pd.Series([1.0, 2.0]).astype('int') == pd.Series([1.0, 2.0002]))
False
>>> all(pd.Series([1.0, np.nan]).dropna().astype('int') == pd.Series([1.0, np.nan]).dropna())
True
```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.