apache / apache/gravitino

[Improvement] Make entity store cache usable in multi-node deployments

Open
#11,737 1 comment 2 reactions 1 assignee Claimed by @yuqi1129 View on GitHub
2.0.0 improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### What would you like to be improved?

The entity store cache (`gravitino.cache.enabled`, default `true`) only invalidates **locally**. In a multi-node deployment it must be set to `false`, which significantly hurts some catalogs (Iceberg especially). The jcasbin cache already solves cross-node invalidation via `entity_change_log` and `EntityChangeLogPoller`; the entity store cache is simply not wired into that channel.

This tracks a design to make `enabled=true` usable in multi-node by invalidating the entity store cache across nodes through the existing change-log mechanism.

### How should we improve?

A design doc is proposed (see linked PR). It covers:

- **Consistency model**: writes stay strongly consistent (DB plus optimistic lock); single-entity and relation reads become cross-node eventually consistent (within one poll interval).
- **The relation-cache difficulty**: relations are cached bidirectionally, so a remote node cannot always derive the reverse keys to invalidate.
- **Two options**: (A, main) coarse invalidation by metalake, a small change that reuses the schema; (B, alternative) precise per-endpoint invalidation, more code but no stampede.
- **Coverage gap**: auth and metadata entities (role/user/group/tag/policy) emit no change-log today and must gain emit points.

Related: #11736 (poller reliability issues found while designing this).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.