delta-io / delta-io/delta-examples
Meta issue: needed example notebooks to demonstrate Delta Lake + other technologies
- Dominant language
- Jupyter Notebook
- Stars
- 242
- Forks
- 83
- PR merge metrics
- No merged PRs in 30d
Description
We need example notebooks that show how to use Delta Lake with a variety of different technologies. The example notebooks should be easily reproducible (have a separate conda environment file that others can run to install the dependencies) and illustrate key features of Delta Lake that make it a good option. For example, a notebook demonstrating a workflow orchestrator should highlight the ACID transaction guarantees offered by Delta Lake.
A ML training notebook should show how Delta Lake allows for file skipping and faster model training.
Here are some key technologies that are top of mind to demonstrate, Delta Lake and...:
* Ibis
* Airflow
* Fugue
* DataFusion
* Prefect
* DuckDB
* Clickhouse
* Airbyte
* Hive
* AWS Glue
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named. Start by reviewing the repository’s existing notebook and conda-environment patterns, then scope one listed integration; done means a reproducible notebook demonstrates a specific Delta Lake benefit and its dependencies install from a separate environment file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, clickhouse, jupyter-notebook
- Domain
- data-engineering, documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100