apache / apache/datafusion-python
Build fails against DataFusion 54.1.0 due to new `schema` field on `RecursiveQuery`
- Dominant language
- Python
- Stars
- 604
- Forks
- 174
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 4
Description
**Describe the bug**
`datafusion-python` does not build against DataFusion 54.1.0. The workspace
dependencies are declared as `version = "54"`, so Cargo is free to resolve them
to 54.1.0, but the crate no longer compiles when it does:
```
error[E0063]: missing field `schema` in initializer of `datafusion::logical_expr::RecursiveQuery`
--> crates/core/src/expr/recursive_query.rs:72:20
|
72 | query: RecursiveQuery {
| ^^^^^^^^^^^^^^ missing `schema`
```
DataFusion 54.1.0 added a `schema: DFSchemaRef` field to
`datafusion_expr::logical_plan::RecursiveQuery` and introduced
`RecursiveQuery::try_new()`, which computes that schema by reconciling the
static and recursive terms. Our `PyRecursiveQuery::new()` builds the struct
with a literal initializer, so the new field breaks it.
**To Reproduce**
With a `Cargo.lock` that resolves DataFusion to 54.1.0 (e.g. a fresh checkout
without the committed lock file, or after `cargo update -p datafusion --precise
54.1.0`):
```sh
cargo check --workspace --all-targets
```
**Expected behavior**
`datafusion-python` builds and tests against DataFusion 54.1.0.
**Additional context**
Because the existing version requirements are `"54"`, this is not limited to
developers who explicitly opt in — any consumer resolving the dependency fresh
picks up 54.1.0 and hits the failure. The fix is to use
`RecursiveQuery::try_new()` and pin the workspace requirements to `"54.1"`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in crates/core/src/expr/recursive_query.rs at the PyRecursiveQuery::new() initializer and inspect the workspace DataFusion dependency requirements. Run cargo check --workspace --all-targets with DataFusion 54.1.0; done means the constructor and dependency requirements support that release and the workspace builds and tests successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100