[Improvement] Provide composite catalog for various catalog
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### What would you like to be improved?
Currently, we maintain separate metadata management systems for Iceberg, Hive, and Paimon, each with its unique cataloging approach. While this separation allows for specialized optimizations, it also leads to increased complexity in the management and usage of metadata across different systems.
Considering for a unified metadata management system, I propose the development of a new component, tentatively named TableCatalog, which can also avoid the issue of data inconsistency.
### How should we improve?
_No response_
Contributor guide
Research direction
The issue names separate metadata systems for Iceberg, Hive, and Paimon but no files, tests, or entry points. Begin by examining how those cataloging systems are currently managed and clarify the TableCatalog scope, consistency requirements, and acceptance criteria before implementation; the issue does not define what done looks like.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100