[Improvement] Group-based authorization adds 2-3x latency overhead compared to user-based authorization
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 339
Description
### What would you like to be improved?
Group-based authorization (built-in IDP with group→role mapping) introduces significant latency overhead compared to direct user-based authorization for single-table operations.
**Performance test results** (30 concurrent users, Auth:ON, Cache:ON, CredentialVending:ON, schema `load3000` with 10,496 tables):
| Service | Auth Mode | Samples | RPS | P50 | P75 | P95 |
|---------|-----------|---------|-----|-----|-----|-----|
| Iceberg REST | User Auth | 4001 | 13.36 | 130ms | 140ms | 180ms |
| Iceberg REST | Group Auth | 3673 | 12.27 | 300ms | 360ms | 510ms |
| Gravitino | User Auth | 4142 | 13.83 | 65ms | 75ms | 110ms |
| Gravitino | Group Auth | 3834 | 12.81 | 210ms | 270ms | 430ms |
Key observations:
- **Group Auth adds ~130-170ms extra latency per request** compared to User Auth on the same operation.
- P50 latency increases by **130% (Iceberg REST)** and **223% (Gravitino)** when switching from User Auth to Group Auth.
- Tail latency (P95) is disproportionately affected — P95/P50 ratio grows from 1.38x to 1.70x (Iceberg) and 1.69x to 2.05x (Gravitino), suggesting occasional cache misses or extra lookups in the group→role resolution path.
- RPS impact is moderate (~8% drop), but the latency increase is significant for latency-sensitive workloads.
The extra latency likely comes from the additional group→role resolution step required on each request when using group-based authorization.
### Test Environment Background
- **Metalake:** `irc_test`
- **Catalog:** `iceberg` (Iceberg REST service at port 9001, Gravitino service at port 8090)
- **Schema:** `load3000` containing **10,496 tables**
- **Concurrent users:** 30 (user1 - user30)
- **Group Auth setup:** All 30 users belong to a single group `perf_test_group`, which is granted `role_all` (full metalake access). The built-in IDP extension (`org.apache.gravitino.idp.web.rest.feature`) is enabled with `gravitino.authenticators = basic`.
- **User Auth setup:** Each user is directly assigned `role_all` plus an individual role (`role1` - `role30`).
- **Operation:** Load single table (random table from `load3000` schema)
### How should we improve?
Potential solutions:
1. **Cache group→role mapping** — Cache the resolution from group membership to role assignments so it does not require a lookup on every request.
2. **Reduce IDP lookup round-trips** — Batch or pre-resolve user→group→role chain at authentication time rather than at each authorization check.
3. **Session-level authorization context** — Resolve the full permission set once per session/token and reuse it for subsequent requests within the same session.
Contributor guide
Assessment
This issue has not been assessed yet.