GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator
Use ibis DuckDB backend to access pandas dataframe instead of ibis pandas backend
Open
type: feature request
- Dominant language
- Python
- Stars
- 524
- Forks
- 171
- Avg merge
- 5d 15h
- Merged PRs (30d)
- 4
Description
**Is your feature request related to a problem? Please describe.**
A feature request - DVT currently uses a pandas backend that has been removed in Ibis 10.0 (we are currently on Ibis 5.1). You can use DuckDB to access data stored in Pandas dataframes. There are some performance benefits and some feature changes. Ibis explains why in this [article](https://ibis-project.org/posts/farewell-pandas). In version 9.0, ibis introduced the concept of a default backend and they chose [DuckDB](https://ibis-project.org/posts/why-duckdb/).
**What version of DVT are you using?**
8.3
Contributor guide
Assessment
This issue has not been assessed yet.