apache / apache/gravitino

[Subtask] Batch tag and policy inheritance relation lookups

Open
#12,152 0 comments 0 reactions 0 assignees View on GitHub
subtask
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Describe the subtask

Avoid query amplification in tag and policy inheritance after relation-list caching is removed by #11961.

Tag and policy lookups inspect the target metadata object and its ancestors. Executing one relation query per ancestor makes database load grow with hierarchy depth, especially for hierarchical schemas.

This subtask should:

- Add a batch relation lookup that accepts the metadata object and its ancestors.
- Return enough source-to-target information to preserve direct and inherited tag/policy semantics.
- Keep the number of backend relation queries bounded independently of hierarchy depth.
- Avoid reintroducing the generic bidirectional relation cache or reverse indexes.
- Validate the SQL and required composite indexes on H2, MySQL, and PostgreSQL.
- Add representative benchmarks covering hierarchy depth, relation cardinality, statement count, p95/p99 latency, and database QPS.
- Record other relation hot spots discovered during the audit as separate follow-ups instead of expanding this subtask indefinitely.

Acceptance criteria:

- Direct and inherited tag/policy results remain unchanged.
- Tests assert that relation-query count does not grow with ancestor depth.
- Database plans use the intended indexes and avoid full scans.
- Benchmark results are attached before merge and compared with the direct-store fallback introduced by #11961.

### Parent issue

#11737

Contributor guide

Open the contributing guide

Research direction

Start by reading the parent issue #11737 and the direct-store fallback introduced by #11961, then trace the existing tag and policy ancestor lookups. Validate the batch SQL and composite-index plans on H2, MySQL, and PostgreSQL, and add tests and benchmarks that confirm unchanged results, depth-independent query counts, and the requested latency and QPS measurements.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.