apache / apache/gravitino

[Subtask] Restrict the entity cache to the entity types that are safe to cache per node

Open
#12,018 0 comments 0 reactions 0 assignees View on GitHub
subtask
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Describe the subtask

Once relation data is removed from the cache (#11961), the cache holds only single-entity entries. This subtask restricts *which* entity types the cache keeps, so that every cached entity is safe to serve from a per-node copy.

Cache the self-contained metadata objects — metalake, catalog, schema, table, topic, view, fileset — plus tag, policy, and the job entity. A stale read of any of these is at worst cosmetic (an old comment, property, or job status), never a wrong pointer. Metalake is cached like the rest: disabling or deleting a metalake is a rare, tenant-level admin action, so the brief window until other nodes refresh (one poll interval) is acceptable, and no special handling is needed for the in-use flag.

Do not cache model, model version, or function: each holds a load-bearing pointer (a model version URI, the latest version, or the function implementation) that would be silently wrong on another node during the staleness window, so they read straight from the `EntityStore`. (A shared cache such as redis has no staleness window and can cache them; that is handled in the redis subtask.) User, group, and role also stay out, because their derived fields (`roleNames`, `securableObjects`) would need a reverse lookup.

This subtask should:

- Add an entity-type allowlist to the cache write path: cache only metalake, catalog, schema, table, topic, view, fileset, tag, policy, and job.
- Skip caching for model, model version, function, user, group, and role; route their reads to the `EntityStore`.
- Add unit tests for the allowlist (cached vs. not-cached types).

The cross-node invalidation for these types, and the change-log emit point that tag, policy, and job still need, are handled in the caffeine multi-node subtask (#12019). This keeps every cached entity safe to serve per node, which is a prerequisite for that work under #11737.

### Parent issue

https://github.com/apache/gravitino/issues/11737

Contributor guide

Open the contributing guide

Research direction

Start by locating the cache write path and the reads that currently use it instead of the EntityStore. Add unit coverage for cached and non-cached entity types, including the listed allowlist and excluded model, user, group, and role types. Done means only safe types are cached and excluded types read from EntityStore; cross-node invalidation and change-log emission are out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.