aws / aws/sagemaker-scikit-learn-container
Update of numpy to version 1.21.0 causing conflict with pyarrow version
Open
bug
- Dominant language
- Python
- Stars
- 188
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I believe the recent upgrade of numpy to version 1.21.0 has caused the below error to occur when converting a dataframe to parquet format (df.to_parquet(f)).
**pyarrow.lib.ArrowTypeError: ('Did not pass numpy.dtype object', 'Conversion failed for column xxxxxxxx with type int64')**
This was fixed locally by upgrading pyarrow to version 3.0.
Would it be possible to upgrade the pyarrow version in the image?
Many thanks.
Contributor guide
Assessment
This issue has not been assessed yet.