apache / apache/datafusion

[Discuss] Move extension point traits out of core?

Open
#2,625 1 comment 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
There are two traits in the core datafusion that could perhaps be moved out into a separate crate.

- VarProvider
- FunctionRegistry

Technically they could move to `datafusion-expr` since they only depend on types from that crate but that doesn't seem like the right place for them. It also doesn't seem to make sense to create a new crate just for two traits.

Moving them out from core would mean:

- `datafusion-proto` would no longer need to depend on the core crate
- `SimplifyExpression` optimizer rule would have one less dependency on the core crate

**Describe the solution you'd like**
Not sure yet but maybe this is the start of a `datafusion-api` crate. Maybe other key traits like `ExecutionPlan` could also move there?

**Describe alternatives you've considered**
None

**Additional context**
None

Contributor guide

Open the contributing guide

Research direction

Start by locating the core definitions of the VarProvider and FunctionRegistry traits and reviewing their dependencies on datafusion-expr. Examine how datafusion-proto and the SimplifyExpression optimizer rule use the core crate, then consider whether a datafusion-api crate should also contain traits such as ExecutionPlan. Done requires an agreed crate boundary and updated dependencies, but the issue does not define that design.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.