Improve numeric inference capabilities Double/Integer/WholeNumber
- Ngôn ngữ chính
- Python
- Star
- 155
- Fork
- 24
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
In the current approach to logical type inference the main differentiator between a `Double` and an `Integer` or `WholeNumber` logical type is the underlying pandas `dtype`. So, in the following examples, these series would be inferred to have a logical type of `Double` when a `WholeNumber` or `Integer` type would be more appropriate.
```python
from woodwork.data_column import infer_logical_type
# inferred as Double because dtype is float64, but could be WholeNumber with Int64 dtype
pd.Series([1.0, 2.0, -3.0])
>>> infer_logical_type(pd.Series([1.0, np.nan, 3.0]))
Double
# inferred as Double because dtype is float64, but could be Integer with Int64 dtype
>>> infer_logical_type(pd.Series([1.0, 2.0, -3.0]))
Double
```
We should improve the inference for series that contain `NaN` values or for series that can be represented as `Integer` or `WholeNumber` types without loss of information. One way to check for this loss of information would be to cast columns inferred as `Double` to integer and then determine if the values are equal to the original float values or not. Would need to drop `NaN` values first to perform this comparison.
```python
>>> all(pd.Series([1.0, 2.0]).astype('int') == pd.Series([1.0, 2.0]))
True
>>> all(pd.Series([1.0, 2.0]).astype('int') == pd.Series([1.0, 2.0002]))
False
>>> all(pd.Series([1.0, np.nan]).dropna().astype('int') == pd.Series([1.0, np.nan]).dropna())
True
```
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.