apache / apache/datafusion

Add subquery support in `datafusion-proto`

Open
#2,640 5 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Once https://github.com/apache/arrow-datafusion/pull/2639 is merged we should add support for serializing plans that contain subquery expressions.

This will likely require breaking changes to the API because we currently have one API for `Expr` that optionally accepts a `FunctionRegistry` and one API for `LogicalPlan` that optionally accepts a `LogicalExtensionCodec`. With subquery support we have plans referencing expressions and expressions referencing plans, so these APIs will need to be combined somehow.

Here is a suggestion for the new API from the discussion at https://github.com/apache/arrow-datafusion/pull/2639#discussion_r884249631

``` rust
let bytes = PlanSerializer::new()
.with_extension_codec(&extension_codec)
.serialize()?;
```

**Describe the solution you'd like**
TBD

**Describe alternatives you've considered**
None

**Additional context**
None

Contributor guide

Open the contributing guide

Research direction

Start by reviewing datafusion-proto's Expr and LogicalPlan serialization APIs, then read PR 2639 and its linked API discussion. Trace how FunctionRegistry and LogicalExtensionCodec are currently passed and determine the combined API needed for expressions referencing plans. Done means plans containing subquery expressions can be serialized through the agreed API.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.