apache / apache/gravitino

[FEATURE] Store metadata of databases and tables into Gravitino stotage

Open
#4,026 0 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 15h
Merged PRs (30d)
315

Description

### Describe the feature

Store more metadata of dbs and tables, such as columns, properties, to gravitino storage.

### Motivation

To provide more metadata of database and table for other systems to use more easily, we can dump the meta from gravitino storage directly instead of invoking the api of gravitino .

But now gravitino doesn't store the columns and properties of table.

### Describe the solution

the extra metadata, such as columns, properties, should be written not only to the backend storage of the catalogs, but also to the gravitino storage.

### Additional context

Since there are both write to rdbms and write to non-rdbms operations, how to ensure transactionality will be a problem.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by tracing how database and table metadata is written to catalog backends and Gravitino storage, then investigate how transactionality could work across RDBMS and non-RDBMS writes. Done means columns and properties are stored in Gravitino storage with a defined consistency approach.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.