apache / apache/gravitino

[Subtask] Add fresh-read semantics for consistency-sensitive entity operations

Open
#12,150 2 comments 0 reactions 0 assignees View on GitHub
subtask
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Describe the subtask

Add an explicit cache-bypass read mode for operations that cannot safely depend on asynchronously invalidated entity data.

The local Caffeine cache provides eventual cross-node freshness. Ordinary metadata reads may accept bounded staleness, but operations that control writes, authorization decisions, external actions, or connector behavior need a fresh value from the underlying `EntityStore`.

This subtask should:

- Add an explicit fresh-read mode for `get` and `batchGet`, or equivalent APIs.
- Ensure a fresh read bypasses both cache lookup and cache refill unless the refill is version-fenced.
- Audit and migrate consistency-sensitive call sites, including:
- metalake usage-state preconditions;
- policy state transitions and policy data used for action decisions;
- job status and cancellation;
- job template execution;
- fileset paths and credentials used for external I/O;
- catalog provider/properties used to create or reload catalog wrappers;
- managed schema/table reads used as write or external-action preconditions.
- Preserve the normal cached path for ordinary metadata reads.
- Add metrics for fresh reads so their cost and usage can be observed.

Acceptance criteria:

- Every audited sensitive call site has an explicit cached-or-fresh decision.
- Tests inject a stale cache entry and verify that sensitive operations use current store data.
- Ordinary read paths continue to use the cache.
- The consistency semantics are documented for callers.

### Parent issue

#11737

Contributor guide

Open the contributing guide

Research direction

Start at the EntityStore-backed get and batchGet entry points and trace their cache lookup and refill behavior. Audit the listed metalake, policy, job, fileset, catalog, and managed schema/table call sites, then add stale-cache tests for sensitive operations and cache-use tests for ordinary reads. Done means each call site has an explicit consistency choice, fresh-read metrics exist, and caller semantics are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.