apache / apache/gravitino

[EPIC] Add Policy Management Support to Gravitino

Open
#7,139 0 comments 5 reactions 1 assignee Claimed by @mchades View on GitHub
epic
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Describe the proposal

Gravitino manages metadata for various data assets - tables, filesets, ML models, and messaging topics. It provides a unified API that works consistently across different data systems. As data volumes grow in organizations, managing governance becomes increasingly challenging, creating the need for effective policy controls.

To address these challenges, we're adding a policy framework to Gravitino. This framework will let users define rules that govern how metadata and the associated data are managed. Here are several common application scenarios for policy:

- **Data Lifecycle Management** - Specify Time-to-Live (TTL) for Filesets, Tables, and Models
- **Table Maintenance Strategy** - Orchestrating data organization strategies, including compaction, expiring snapshots, and removing orphan files
- **Access Control Enhancement** - Extending basic permissions with conditional, and temporal access constraints
- **Data Compliance** - Establishing validation frameworks with configurable quality thresholds and remediation actions

### Task list

see below

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.