apache / apache/hamilton

data quality dask series schema issue

Open
#1,279 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Jupyter Notebook
Stars
2.6k
Forks
213
PR merge metrics
No merged PRs in 30d

Description

There is some import ordering issue (I think) that is leading to pandera + dask + hamilton for pandera series schemas to not work properly.
Doing things outside of Hamilton seems to work as expected, but with Hamilton for Series schemas the backend isn't found for pandera correctly.

# Current behavior
It will throw an exception if you try to validate a pandera series schema with a dask series object.

## Stack Traces
Key error saying `dask` isn't a valid backend for pandera. Instead it wants `dask_expr`.

## Steps to replicate behavior
1. Run the pandera integration test and find the one marked with `xfail`.

## Library & System Information

# Expected behavior
This should pass.

# Additional context
We'll likely mark this as a P3 unless someone comes forward to mark this higher.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.