[Improvement] Add a context-aware security contract for entity properties
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### What would you like to be improved?
Gravitino does not have a shared security contract connecting a property's canonical definition to every server-side consumer of that property.
Entity properties are generally represented as `Map`. The current metadata model can mark a `PropertyEntry` as `hidden`, but `PropertiesMetadata.isHiddenProperty()` only protects a key when the consuming metadata includes a matching exact or prefix definition. If no definition is selected, the property follows ordinary compatibility behavior and remains visible.
Fileset catalogs demonstrate this gap. Provider metadata classifies properties such as `s3-secret-access-key`, but Fileset catalog metadata does not select the shared S3/GCS/Azure/OSS definitions. The hidden-property lookup therefore cannot consult the provider's classification, allowing an inline credential to cross a normal entity response boundary.
This is broader than one Fileset property. Property maps are consumed by REST responses, events, errors, logs, connector runtime paths, and other server-side integrations. Every consumer currently has to select the correct metadata and independently apply the intended handling.
Entity Secrets addresses a complementary problem. It provides opt-in secret bindings/references, persistence, and resolution. It does not classify legacy inline values or define how all property representations should be handled across different runtime and disclosure contexts.
Gravitino should provide an additive property-security contract that:
- lets the canonical property definition declare security semantics once;
- keeps classification independent of inline or Entity Secrets-backed storage;
- provides deterministic safe projections for API and observer boundaries;
- provides selected material access for trusted runtime operations;
- preserves compatibility for existing ordinary and `hidden` properties; and
- supports incremental migration by property and consumer.
The first vertical slice should apply the contract to Fileset cloud-provider properties and close #11642.
### How should we improve?
Introduce an additive, server-side property contract with three cooperating parts:
1. **Property definition**
Extend canonical `PropertyEntry` definitions with reviewed security profiles such as `ORDINARY`, `SENSITIVE`, `SECRET`, and `INTERNAL`, plus an optional coarse semantic descriptor such as `OPAQUE`.
Existing `hidden=true` declarations should retain their current behavior through an internal legacy compatibility profile. Other existing and undeclared custom properties should continue to behave as ordinary properties unless explicitly classified.
2. **Context-aware access decision**
Bind loaded property values to the definitions explicitly selected by the consumer. Evaluate each property using a structured context containing a typed purpose and surface, with named presets for common Gravitino operations.
The decision should produce a small set of actions such as:
- expose the stored value;
- omit the property;
- return a fixed, content-independent replacement;
- report presence without material; or
- permit selected runtime materialization.
This context describes how the property is being used. It is not a new authorization or per-field ACL system.
3. **Default execution library**
Provide a core projector that executes these actions and returns an immutable, context-specific result suitable for serialization or observation. Consumers should not reimplement masking or filtering.
Projection must be pure: it must never resolve Entity Secrets handles or fall back to returning raw values when selected metadata cannot be evaluated.
Trusted runtime consumers should use a separate selected-key materialization operation. Inline material can be supplied directly, while recognized Entity Secrets handles can be resolved through an adapter to the existing `SecretManager`. Existing `getSecrets()` behavior should remain unchanged until its owner reviews any broader integration.
Implementation should proceed as independently valid vertical slices. The first slice should:
- compose the applicable S3, GCS, Azure, and OSS definitions for Fileset catalogs;
- review each provider property's classification rather than assuming every provider value is secret;
- apply the shared projection to Fileset create, load, and list responses;
- preserve the existing fixed replacement and write-rejection compatibility behavior for legacy hidden properties; and
- verify that trusted connector runtime configuration still receives the required material.
#### Acceptance criteria
- A canonical property definition can declare one reviewed security profile independently of its storage representation.
- Ordinary projections never return classified secret material or Entity Secrets handles.
- Secret replacement is fixed and content-independent; it does not reveal the value's length or structure.
- Presence, omission, replacement, absence, and projection failure remain distinguishable states.
- Selected metadata composition failures produce value-free internal errors rather than raw-map fallback.
- Undeclared custom properties retain ordinary compatibility behavior.
- Legacy `hidden=true` properties retain their existing outward behavior while supporting incremental migration.
- The Fileset S3 reproduction is covered by server-side integration tests.
- Tests cover inline secrets, recognized Entity Secrets handles, legacy hidden properties, undeclared properties, metadata composition failures, create/load/list responses, and trusted runtime access.
#### Out of scope
- Per-principal property ACLs or new authorization rules
- Requiring all secrets to use Secret Manager
- Changing the public `getSecrets()` authorization or response contract
- Protecting the metadata database, backups, or historical disclosures
- Relying on UI or client-side redaction as the security boundary
#### Related work
- #11642 — Fileset catalog provider-property metadata gap
- #11644 — Earlier connector-specific credential exposure
- #12252 — Entity Secrets SPI, REST contract, and in-memory provider
- #12457 — `getSecrets()` for catalog, schema, and Fileset connection secrets
Contributor guide
Assessment
This issue has not been assessed yet.