[FEATURE] Store metadata of databases and tables into Gravitino stotage
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 315
Description
### Describe the feature
Store more metadata of dbs and tables, such as columns, properties, to gravitino storage.
### Motivation
To provide more metadata of database and table for other systems to use more easily, we can dump the meta from gravitino storage directly instead of invoking the api of gravitino .
But now gravitino doesn't store the columns and properties of table.
### Describe the solution
the extra metadata, such as columns, properties, should be written not only to the backend storage of the catalogs, but also to the gravitino storage.
### Additional context
Since there are both write to rdbms and write to non-rdbms operations, how to ensure transactionality will be a problem.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by tracing how database and table metadata is written to catalog backends and Gravitino storage, then investigate how transactionality could work across RDBMS and non-RDBMS writes. Done means columns and properties are stored in Gravitino storage with a defined consistency approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100