[Improvement] Metrics sent to Iceberg Metrics REST API currently dropped. Expose Iceberg client metrics through Gravitino MetricsSystem
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### What would you like to be improved?
Currently, when the Iceberg REST server receives metrics from clients via the
`/v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics` endpoint, they are
silently dropped if metricsStore is set to "dummy" (the default). This provides
no observability into:
- Iceberg table operations (commits, scans)
- Data file operations (added/removed files, sizes)
- Query performance and patterns
We need a way to expose these metrics for monitoring and debugging.
### How should we improve?
1. Add IcebergRestMetricsStore that implements Iceberg's MetricsStore interface
and bridges metrics to Gravitino's MetricsSystem
2. Add IcebergClientMetricsSource with "iceberg-client" namespace (separate from
"iceberg-rest-server" HTTP metrics to avoid collision)
3. Parse commit and scan reports using Iceberg's public APIs (CommitReportParser,
ScanReportParser)
4. Expose metrics via the existing /metrics endpoint under iceberg-client namespace
5. Allow configuration via: gravitino.iceberg-rest.metricsStore = rest
This will enable production observability of Iceberg operations through Gravitino's
existing metrics infrastructure.
Contributor guide
Assessment
This issue has not been assessed yet.