apache / apache/datafusion

FFI schema providers fail when views are registered

Open
#21,155 4 comments 0 reactions 1 assignee Claimed by @timsaucer View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

When you have a FFI based schema provider and you register a view with it, it is unable to be executed. This is a regression from 51.0.0 to 52.3.0.

### To Reproduce

Add this unit test to `datafusion/ffi/tests/ffi_catalog.rs`

```
#[tokio::test]
async fn test_view_on_schema_provider() -> datafusion_common::Result<()> {
let module = get_module()?;
let (ctx, codec) = super::utils::ctx_and_codec();

let ffi_catalog =
module
.create_catalog()
.ok_or(DataFusionError::NotImplemented(
"External catalog provider failed to implement create_catalog"
.to_string(),
))?(codec);
let foreign_catalog: Arc = (&ffi_catalog).into();

let _ = ctx.register_catalog("fruit", foreign_catalog);

let df = ctx.table("fruit.apple.purchases").await?;

let df_view = df.into_view();

let _ = ctx
.catalog("fruit")
.unwrap()
.schema("apple")
.unwrap()
.register_table("sales".to_owned(), df_view);

let df = ctx.table("fruit.apple.sales").await?;
let results = df.collect().await?;

assert_eq!(results.len(), 2);
let num_rows: usize = results.into_iter().map(|rb| rb.num_rows()).sum();
assert_eq!(num_rows, 5);

Ok(())
}
```

### Expected behavior

These views should work.

### Additional context

Originally filed as https://github.com/apache/datafusion-python/issues/1444

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.