apache / apache/gravitino

[Improvement] Support single-flight for concurrent read requests on the same Iceberg table

Open
#12,502 1 comment 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### What would you like to be improved?

Under high concurrency, concurrent reads targeting the same Iceberg table can overwhelm the underlying storage when the cache is cold or expired. N concurrent requests may trigger N identical upstream loads, causing redundant I/O, latency spikes, and unnecessary backend load.

The current read path is:

```
IcebergTableOperations.loadTable / planTableScan
└─ IcebergCatalogWrapper.loadTable / CatalogWrapperForREST.planTableScan
├─ TableMetadataCache / ScanPlanCache: getIfPresent + put (check-then-act)
└─ cache miss → CatalogHandlers.loadTable / scan planning → underlying catalog → OSS/metadata store
```

Key observations:

1. `TableMetadataCache` (`iceberg-common/.../cache/LocalTableMetadataCache.java`) uses `getIfPresent` followed by `put`, rather than Caffeine's atomic `get(key, mappingFunction)`. Concurrent misses therefore invoke `IcebergCatalogWrapper.loadTable` independently and read the same metadata files multiple times.
2. `ScanPlanCache` (`iceberg-rest-server/.../cache/LocalScanPlanCache.java`) uses the same check-then-act pattern and is disabled by default (`ScanPlanCache.DUMMY`). In addition, `planTableScan` (`iceberg-rest-server/.../CatalogWrapperForREST.java`) calls `getCatalog().loadTable()` before consulting the cache.

As a result, identical concurrent requests are not coalesced.

### How should we improve?

Introduce single-flight semantics into the read path:

1. **`TableMetadataCache` and `ScanPlanCache`**: replace `getIfPresent` + `put` with a single-flight load, such as Caffeine's `get(key, loader)`, so only one loader runs per key and concurrent callers share the result. Propagate loader failures to all waiting callers.
2. Optionally extract a reusable single-flight helper, using shared `Future`/`CompletableFuture` instances or a thin Caffeine wrapper, with a configurable per-key timeout.
3. Move `getCatalog().loadTable()` after the scan-plan cache lookup, or reuse `TableMetadataCache`, so scan-plan cache hits do not access the underlying catalog.
4. Add unit tests for:
- Concurrent misses on the same key: only one upstream call.
- Concurrent misses on different keys: no cross-key blocking.
- Failure propagation to all waiting callers.

Contributor guide

Open the contributing guide

Research direction

Start with LocalTableMetadataCache.java, LocalScanPlanCache.java, IcebergCatalogWrapper.loadTable, and CatalogWrapperForREST.planTableScan. Trace the current cache-miss flow, then run the existing cache and REST tests before adding coverage for same-key coalescing, different-key concurrency, and shared failures. Done means concurrent identical reads make one upstream call and scan-plan cache hits avoid unnecessary catalog loads.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
67/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.