[EPIC] Add Credential Management
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### Describe the proposal
Problem
Currently, Gravitino lacks a centralized and secure system for managing credentials (e.g., passwords, access keys, tokens) needed to connect to and interact with underlying data sources (like RDBMS, data lakes, message queues). This leads to:
Security Risks: Credentials may be stored in plaintext within catalog properties or configuration files.
Operational Overhead: Manual credential rotation is cumbersome and error-prone across multiple catalogs.
Lack of Audit Trail: No centralized way to track who accessed or modified which credential.
Goal
Integrate a secure Credential Vault into Gravitino to centrally store, manage, and automatically inject credentials for catalog connections and other internal operations.
Core Requirements
Secure Storage: Support integration with external secret managers (e.g., HashiCorp Vault, AWS Secrets Manager, KMS) as backends. Provide a secure internal storage option for simplicity.
Lifecycle Management: APIs/UI to create, read, update, rotate, and delete credentials.
Catalog Integration: Allow catalog properties to reference a credential stored in the vault (e.g., password: {{vault://my-secret/password}}) instead of containing the actual secret value.
Access Control: Fine-grained access control to determine which users/principals can read or manage specific credentials.
Audit Logging: All access and modifications to credentials must be logged.
### Task list
- [ ] #4681
- [x] #10080
- [ ] #9643
- [ ] #10417
Contributor guide
Research direction
Start by reviewing the task list items #4681, #10080, #9643, and #10417, since the issue names no implementation files or tests. The work is complete only when the listed credential storage, lifecycle, catalog integration, access control, and audit requirements have been resolved across the relevant follow-up tasks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- authentication, authorization, backend-api-design, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100