apache / apache/datafusion-python

Build fails against DataFusion 54.1.0 due to new `schema` field on `RecursiveQuery`

Open Beginner friendly
#1,644 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
604
Forks
174
Avg merge
1d 7h
Merged PRs (30d)
4

Description

**Describe the bug**

`datafusion-python` does not build against DataFusion 54.1.0. The workspace
dependencies are declared as `version = "54"`, so Cargo is free to resolve them
to 54.1.0, but the crate no longer compiles when it does:

```
error[E0063]: missing field `schema` in initializer of `datafusion::logical_expr::RecursiveQuery`
--> crates/core/src/expr/recursive_query.rs:72:20
|
72 | query: RecursiveQuery {
| ^^^^^^^^^^^^^^ missing `schema`
```

DataFusion 54.1.0 added a `schema: DFSchemaRef` field to
`datafusion_expr::logical_plan::RecursiveQuery` and introduced
`RecursiveQuery::try_new()`, which computes that schema by reconciling the
static and recursive terms. Our `PyRecursiveQuery::new()` builds the struct
with a literal initializer, so the new field breaks it.

**To Reproduce**

With a `Cargo.lock` that resolves DataFusion to 54.1.0 (e.g. a fresh checkout
without the committed lock file, or after `cargo update -p datafusion --precise
54.1.0`):

```sh
cargo check --workspace --all-targets
```

**Expected behavior**

`datafusion-python` builds and tests against DataFusion 54.1.0.

**Additional context**

Because the existing version requirements are `"54"`, this is not limited to
developers who explicitly opt in — any consumer resolving the dependency fresh
picks up 54.1.0 and hits the failure. The fix is to use
`RecursiveQuery::try_new()` and pin the workspace requirements to `"54.1"`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in crates/core/src/expr/recursive_query.rs at the PyRecursiveQuery::new() initializer and inspect the workspace DataFusion dependency requirements. Run cargo check --workspace --all-targets with DataFusion 54.1.0; done means the constructor and dependency requirements support that release and the workspace builds and tests successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
build-system
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.