delta-io / delta-io/delta-examples

Meta issue: needed example notebooks to demonstrate Delta Lake + other technologies

Open
#6 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
242
Forks
83
PR merge metrics
No merged PRs in 30d

Description

We need example notebooks that show how to use Delta Lake with a variety of different technologies. The example notebooks should be easily reproducible (have a separate conda environment file that others can run to install the dependencies) and illustrate key features of Delta Lake that make it a good option. For example, a notebook demonstrating a workflow orchestrator should highlight the ACID transaction guarantees offered by Delta Lake.

A ML training notebook should show how Delta Lake allows for file skipping and faster model training.

Here are some key technologies that are top of mind to demonstrate, Delta Lake and...:

* Ibis
* Airflow
* Fugue
* DataFusion
* Prefect
* DuckDB
* Clickhouse
* Airbyte
* Hive
* AWS Glue

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by reviewing the repository’s existing notebook and conda-environment patterns, then scope one listed integration; done means a reproducible notebook demonstrates a specific Delta Lake benefit and its dependencies install from a separate environment file.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, clickhouse, jupyter-notebook
Domain
data-engineering, documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.