aws / aws/amazon-sagemaker-examples

[Bug Report] xgboost_customer_churn.ipynb fails

Open
#3,069 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

**Link to the notebook**
https://github.com/aws/amazon-sagemaker-examples/blob/master/introduction_to_applying_machine_learning/xgboost_customer_churn/xgboost_customer_churn.ipynb

**Describe the bug**
Running Cell [7]:
```
# Frequency tables for each categorical feature
for column in churn.select_dtypes(include=["object"]).columns:
display(pd.crosstab(index=churn[column], columns="% observations", normalize="columns"))

# Histograms for each numeric features
display(churn.describe())
%matplotlib inline
hist = churn.hist(bins=30, sharey=True, figsize=(10, 10))
```
The result is the following error:
```
TypeError: Cannot interpret '' as a data type
```

RCA: [pandas <1.0.5 is not compatible with numpy 1.20](https://github.com/hdmf-dev/hdmf/issues/617)
On SageMaker Studio - datascience-1-0-ml-t3-medium:
`pandas==1.0.1`
`numpy==1.20.3`

**To reproduce**
Run the notebook.

**Logs**
```
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in
1 # Frequency tables for each categorical feature
2 for column in churn.select_dtypes(include=["object"]).columns:
----> 3 display(pd.crosstab(index=churn[column], columns="% observations", normalize="columns"))
4
5 # Histograms for each numeric features

/opt/conda/lib/python3.7/site-packages/pandas/core/reshape/pivot.py in crosstab(index, columns, values, rownames, colnames, aggfunc, margins, margins_name, dropna, normalize)
579 from pandas import DataFrame
580
--> 581 df = DataFrame(data, index=common_idx)
582 if values is None:
583 df["__dummy__"] = 0

/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py in __init__(self, data, index, columns, dtype, copy)
433 )
434 elif isinstance(data, dict):
--> 435 mgr = init_dict(data, index, columns, dtype=dtype)
436 elif isinstance(data, ma.MaskedArray):
437 import numpy.ma.mrecords as mrecords

/opt/conda/lib/python3.7/site-packages/pandas/core/internals/construction.py in init_dict(data, index, columns, dtype)
252 arr if not is_datetime64tz_dtype(arr) else arr.copy() for arr in arrays
253 ]
--> 254 return arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
255
256

/opt/conda/lib/python3.7/site-packages/pandas/core/internals/construction.py in arrays_to_mgr(arrays, arr_names, index, columns, dtype)
67
68 # don't force copy because getting jammed in an ndarray anyway
---> 69 arrays = _homogenize(arrays, index, dtype)
70
71 # from BlockManager perspective

/opt/conda/lib/python3.7/site-packages/pandas/core/internals/construction.py in _homogenize(data, index, dtype)
321 val = lib.fast_multiget(val, oindex.values, default=np.nan)
322 val = sanitize_array(
--> 323 val, index, dtype=dtype, copy=False, raise_cast_failure=False
324 )
325

/opt/conda/lib/python3.7/site-packages/pandas/core/construction.py in sanitize_array(data, index, dtype, copy, raise_cast_failure)
463 value = maybe_cast_to_datetime(value, dtype)
464
--> 465 subarr = construct_1d_arraylike_from_scalar(value, len(index), dtype)
466
467 else:

/opt/conda/lib/python3.7/site-packages/pandas/core/dtypes/cast.py in construct_1d_arraylike_from_scalar(value, length, dtype)
1459 value = ensure_str(value)
1460
-> 1461 subarr = np.empty(length, dtype=dtype)
1462 subarr.fill(value)
1463

TypeError: Cannot interpret '' as a data type
```

Contributor guide

Open the contributing guide

Research direction

Open introduction_to_applying_machine_learning/xgboost_customer_churn/xgboost_customer_churn.ipynb and run Cell [7] in the reported SageMaker Studio environment. Check the pandas==1.0.1 and numpy==1.20.3 compatibility described in the report; done means the cell runs without the reported TypeError and still displays the frequency tables, summary, and histograms.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, numpy, pandas
Domain
data, machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.