apache / apache/datafusion

Custom CLI Mode with Manual ```import``` for Functions

Open
#14,588 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

As we increase the number of functions in our core, it might lead to an increased runtime footprint for datafusion-cli in the future. This increase will be due to functions that a lot of users with specific use-cases in mind wouldn't need. The same can be claimed for users using the cli for a more general purpose.

### Describe the solution you'd like

Adding a new feature wherein a user pass a flag while starting the ```datafusion-cli``` that will start the CLI with a "bare-bones" set of functions. These functions shall be the absolutely necessary ones. Then the user shall manually import the set of functions as needed.

This feature primarily aims to provide users with more customization along with a reduced runtime footprint.

### Describe alternatives you've considered

_No response_

### Additional context

This idea originated from the discussion on creating a separate crate for spark functions. The idea of having an import command was first mentioned there. This issue proposes to provide the import command in context of a custom cli.

The original comment: https://github.com/apache/datafusion/issues/5600#issuecomment-2648452637

Contributor guide

Open the contributing guide

Research direction

Start with the datafusion-cli entry point and read the linked discussion about separating Spark functions and introducing an import command. Define the scope of the custom startup mode, the minimal function set, and manual function importing before implementation. Done means the CLI can start in the custom mode and users can import the functions they need.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.