apache / apache/gravitino

[FEATURE] Add a JDBC catalog of DuckLake

Open
#7,313 7 comments 1 reaction 0 assignees View on GitHub
feature
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Describe the feature

**Describe the feature**

DuckDB has just released an intuitive data lake format and catalog specification for running DuckDB as a lakehouse. The catalog specification is backed by a SQL instance for metadata management (as opposed to storage based metadata techniques found in Iceberg, Delta etc..).

**Motivation**

DuckDB is a simple and effective database. More users use it to analyze data. DuckLake is a catalog specification and table format that is open and accessible for different compute engines.

**Describe the solution**

Implement catalog actions by leveraging the database backend of DuckLake which is all sql based.

**Additional context**

For more information see:
- https://ducklake.select/
- https://ducklake.select/docs/stable/specification/introduction

Contributor guide

Open the contributing guide

Research direction

Start by reading the DuckLake specification at ducklake.select, especially its introduction, and determine the catalog actions required for a SQL-backed metadata catalog. Then inspect Gravitino's existing catalog implementations and JDBC-related entry points. Done means DuckLake catalog actions are implemented against the SQL database backend and covered by appropriate tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.