apache / apache/datafusion

Library Guide: Add SQL level user guide:

Open
#7,302 4 comments 0 reactions 0 assignees View on GitHub
documentation enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

Part of https://github.com/apache/arrow-datafusion/issues/7014

If we want to have DataFusion used as the core of many new systems, we need it to be as easy as possible for someone to get their idea working on top of DataFusion.

Thanks to @tshauck we now have a basic `Library Users Guide` ❤️ and this ticket describes expanding it out

### Describe the solution you'd like

Fill in the content of https://arrow.apache.org/datafusion/library-user-guide/using-the-sql-api.html

Example outline

1. Generating LogicalPlans from `sql`
2. Example / guide of how to extend SQL (both links to user defined functions and custom nodes, but also maybe some links to the datafusion sql variant parser
3. Example of how to run SQL read only and how to run it with catalog modifications (https://github.com/apache/arrow-datafusion/issues/7272)

The examples directory holds a bunch of examples: https://github.com/apache/arrow-datafusion/tree/main/datafusion-examples

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the existing page at arrow.apache.org/datafusion/library-user-guide/using-the-sql-api.html and review the examples in datafusion-examples. Cover generating LogicalPlans from SQL, extending SQL with user-defined functions or custom nodes, and running read-only SQL and catalog modifications, including the linked issue 7272. Done means the SQL API guide is filled in with working, discoverable examples and relevant links.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.