Add `include_fully_null_columns` parameter to mutual info to allow for more control
- 主要语言
- Python
- 星标
- 155
- 派生
- 24
- PR 合并指标
- 30 天内没有已合并 PR
描述
We should add the following parameter to the mutual info function and associated defaults.
- include_fully_null_columns=True
This way the end-user can have more control of the logic for calculating MI
This is for the current DataTable implementation
Note - there's behavior that needs to be decided:
A fully null column still needs to be a LogicalType that’s relevant for mutual info calculations (so a Natural Language column of nans still wouldn’t be included), and if no Logical Type is specified, it’ll get inferred as Categorical, which is fine. But now we have two problems:
1. If the column is not categorical, we’ll try and bin the values to make it categorical with pd.qcut. How should this work for a column of nulls if their logical type is, say, Integer or Datetime?
2. We replace null values with the average for numerical columns and the mode for other columns, and the _get_mode function we use will return None for a column of nulls--kind of defeats the purpose here. How should we handle this?
贡献指南
评估
这个 Issue 还没有评估数据。