[Subtask] Add fresh-read semantics for consistency-sensitive entity operations
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### Describe the subtask
Add an explicit cache-bypass read mode for operations that cannot safely depend on asynchronously invalidated entity data.
The local Caffeine cache provides eventual cross-node freshness. Ordinary metadata reads may accept bounded staleness, but operations that control writes, authorization decisions, external actions, or connector behavior need a fresh value from the underlying `EntityStore`.
This subtask should:
- Add an explicit fresh-read mode for `get` and `batchGet`, or equivalent APIs.
- Ensure a fresh read bypasses both cache lookup and cache refill unless the refill is version-fenced.
- Audit and migrate consistency-sensitive call sites, including:
- metalake usage-state preconditions;
- policy state transitions and policy data used for action decisions;
- job status and cancellation;
- job template execution;
- fileset paths and credentials used for external I/O;
- catalog provider/properties used to create or reload catalog wrappers;
- managed schema/table reads used as write or external-action preconditions.
- Preserve the normal cached path for ordinary metadata reads.
- Add metrics for fresh reads so their cost and usage can be observed.
Acceptance criteria:
- Every audited sensitive call site has an explicit cached-or-fresh decision.
- Tests inject a stale cache entry and verify that sensitive operations use current store data.
- Ordinary read paths continue to use the cache.
- The consistency semantics are documented for callers.
### Parent issue
#11737
Contributor guide
Research direction
Start at the EntityStore-backed get and batchGet entry points and trace their cache lookup and refill behavior. Audit the listed metalake, policy, job, fileset, catalog, and managed schema/table call sites, then add stale-cache tests for sensitive operations and cache-use tests for ordinary reads. Done means each call site has an explicit consistency choice, fresh-read metrics exist, and caller semantics are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100