GoogleCloudPlatform / GoogleCloudPlatform/python-docs-samples

Columns and DataType Not Explicitly Set on line 62 of data_utils.py

未关闭
#10,949 2 条评论 0 个 reaction 已指派 1 人 已指派给 @davidcavazos 在 GitHub 查看
samples
主要语言
Jupyter Notebook
星标
8.1k
派生
6.7k
平均合并
4 天 2 小时
30 天内合并 PR
16

描述

Hello!

I found an AI-Specific Code smell in your project.
The smell is called: Columns and DataType Not Explicitly Set

You can find more information about it in this paper: https://dl.acm.org/doi/abs/10.1145/3522664.3528620.

According to the paper, the smell is described as follows:

| **Problem** | If the columns are not selected explicitly, it is not easy for developers to know what to expect in the downstream data schema. If the datatype is not set explicitly, it may silently continue the next step even though the input is unexpected, which may cause errors later. The same applies to other data importing scenarios. |
| ------------- | :------------- |
| **Solution** | **It is recommended to set the columns and DataType explicitly in data processing.** |
| **Impact** | **Readability** |

Example:
```diff

### Pandas Column Selection
import pandas as pd
df = pd.read_csv('data.csv')
+ df = df[['col1', 'col2', 'col3']]

### Pandas Set DataType
import pandas as pd
- df = pd.read_csv('data.csv')
+ df = pd.read_csv('data.csv', dtype={'col1': 'str', 'col2': 'int', 'col3': 'float'})

```
You can find the code related to this smell in this link: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/161448f276cd6fbe697c83e2ec55c4828e9143fa/people-and-planet-ai/timeseries-classification/data_utils.py#L52-L72.

I also found instances of this smell in other files, such as:

File: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/composer/2022_airflow_summit/data_analytics_process_expansion.py#L140-L150 Line: 145
.

I hope this information is helpful!

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。