apache / apache/gravitino

[Improvement] Add a context-aware security contract for entity properties

Open
#12,677 0 comments 0 reactions 1 assignee Claimed by @nevzheng View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### What would you like to be improved?

Gravitino does not have a shared security contract connecting a property's canonical definition to every server-side consumer of that property.

Entity properties are generally represented as `Map`. The current metadata model can mark a `PropertyEntry` as `hidden`, but `PropertiesMetadata.isHiddenProperty()` only protects a key when the consuming metadata includes a matching exact or prefix definition. If no definition is selected, the property follows ordinary compatibility behavior and remains visible.

Fileset catalogs demonstrate this gap. Provider metadata classifies properties such as `s3-secret-access-key`, but Fileset catalog metadata does not select the shared S3/GCS/Azure/OSS definitions. The hidden-property lookup therefore cannot consult the provider's classification, allowing an inline credential to cross a normal entity response boundary.

This is broader than one Fileset property. Property maps are consumed by REST responses, events, errors, logs, connector runtime paths, and other server-side integrations. Every consumer currently has to select the correct metadata and independently apply the intended handling.

Entity Secrets addresses a complementary problem. It provides opt-in secret bindings/references, persistence, and resolution. It does not classify legacy inline values or define how all property representations should be handled across different runtime and disclosure contexts.

Gravitino should provide an additive property-security contract that:

- lets the canonical property definition declare security semantics once;
- keeps classification independent of inline or Entity Secrets-backed storage;
- provides deterministic safe projections for API and observer boundaries;
- provides selected material access for trusted runtime operations;
- preserves compatibility for existing ordinary and `hidden` properties; and
- supports incremental migration by property and consumer.

The first vertical slice should apply the contract to Fileset cloud-provider properties and close #11642.

### How should we improve?

Introduce an additive, server-side property contract with three cooperating parts:

1. **Property definition**

Extend canonical `PropertyEntry` definitions with reviewed security profiles such as `ORDINARY`, `SENSITIVE`, `SECRET`, and `INTERNAL`, plus an optional coarse semantic descriptor such as `OPAQUE`.

Existing `hidden=true` declarations should retain their current behavior through an internal legacy compatibility profile. Other existing and undeclared custom properties should continue to behave as ordinary properties unless explicitly classified.

2. **Context-aware access decision**

Bind loaded property values to the definitions explicitly selected by the consumer. Evaluate each property using a structured context containing a typed purpose and surface, with named presets for common Gravitino operations.

The decision should produce a small set of actions such as:

- expose the stored value;
- omit the property;
- return a fixed, content-independent replacement;
- report presence without material; or
- permit selected runtime materialization.

This context describes how the property is being used. It is not a new authorization or per-field ACL system.

3. **Default execution library**

Provide a core projector that executes these actions and returns an immutable, context-specific result suitable for serialization or observation. Consumers should not reimplement masking or filtering.

Projection must be pure: it must never resolve Entity Secrets handles or fall back to returning raw values when selected metadata cannot be evaluated.

Trusted runtime consumers should use a separate selected-key materialization operation. Inline material can be supplied directly, while recognized Entity Secrets handles can be resolved through an adapter to the existing `SecretManager`. Existing `getSecrets()` behavior should remain unchanged until its owner reviews any broader integration.

Implementation should proceed as independently valid vertical slices. The first slice should:

- compose the applicable S3, GCS, Azure, and OSS definitions for Fileset catalogs;
- review each provider property's classification rather than assuming every provider value is secret;
- apply the shared projection to Fileset create, load, and list responses;
- preserve the existing fixed replacement and write-rejection compatibility behavior for legacy hidden properties; and
- verify that trusted connector runtime configuration still receives the required material.

#### Acceptance criteria

- A canonical property definition can declare one reviewed security profile independently of its storage representation.
- Ordinary projections never return classified secret material or Entity Secrets handles.
- Secret replacement is fixed and content-independent; it does not reveal the value's length or structure.
- Presence, omission, replacement, absence, and projection failure remain distinguishable states.
- Selected metadata composition failures produce value-free internal errors rather than raw-map fallback.
- Undeclared custom properties retain ordinary compatibility behavior.
- Legacy `hidden=true` properties retain their existing outward behavior while supporting incremental migration.
- The Fileset S3 reproduction is covered by server-side integration tests.
- Tests cover inline secrets, recognized Entity Secrets handles, legacy hidden properties, undeclared properties, metadata composition failures, create/load/list responses, and trusted runtime access.

#### Out of scope

- Per-principal property ACLs or new authorization rules
- Requiring all secrets to use Secret Manager
- Changing the public `getSecrets()` authorization or response contract
- Protecting the metadata database, backups, or historical disclosures
- Relying on UI or client-side redaction as the security boundary

#### Related work

- #11642 — Fileset catalog provider-property metadata gap
- #11644 — Earlier connector-specific credential exposure
- #12252 — Entity Secrets SPI, REST contract, and in-memory provider
- #12457 — `getSecrets()` for catalog, schema, and Fileset connection secrets

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.