[Improvement] Make entity store cache usable in multi-node deployments
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### What would you like to be improved?
The entity store cache (`gravitino.cache.enabled`, default `true`) only invalidates **locally**. In a multi-node deployment it must be set to `false`, which significantly hurts some catalogs (Iceberg especially). The jcasbin cache already solves cross-node invalidation via `entity_change_log` and `EntityChangeLogPoller`; the entity store cache is simply not wired into that channel.
This tracks a design to make `enabled=true` usable in multi-node by invalidating the entity store cache across nodes through the existing change-log mechanism.
### How should we improve?
A design doc is proposed (see linked PR). It covers:
- **Consistency model**: writes stay strongly consistent (DB plus optimistic lock); single-entity and relation reads become cross-node eventually consistent (within one poll interval).
- **The relation-cache difficulty**: relations are cached bidirectionally, so a remote node cannot always derive the reverse keys to invalidate.
- **Two options**: (A, main) coarse invalidation by metalake, a small change that reuses the schema; (B, alternative) precise per-endpoint invalidation, more code but no stampede.
- **Coverage gap**: auth and metadata entities (role/user/group/tag/policy) emit no change-log today and must gain emit points.
Related: #11736 (poller reliability issues found while designing this).
Contributor guide
Assessment
This issue has not been assessed yet.