Filebrowser: Add Apache Iceberg table support to S3 file browser

We already support browsing S3 and previewing individual Parquet files using hyparquet.
This issue tracks adding Apache Iceberg table support, so users can:

• Discover Iceberg tables in an S3 bucket
• Browse a table’s metadata (schema, partitions, snapshots)
• Select a snapshot/partition and preview underlying Parquet data files

Background

Apache Iceberg is a table format that sits on top of Parquet (and other) data files. The Parquet files themselves are unchanged; Iceberg adds a metadata layer:

• metadata.json – current table metadata (schema, partition specs, snapshot list, current snapshot ID)
• Manifest list (Avro) – one per snapshot; lists all manifest files for that snapshot
• Manifest files (Avro) – list data/delete files (Parquet) with stats (row count, min/max, partition values)
• Data files – regular Parquet files that our existing preview component can already render

References:

• Iceberg spec: https://iceberg.apache.org/spec/
• Architecture overview: https://iceberglakehouse.com/apache-iceberg-architecture/

Our current Parquet preview can already show schema + data for any Parquet file. For Iceberg, we need to:

1. Detect Iceberg tables in S3
2. Parse Iceberg metadata (JSON + Avro manifest list/manifests)
3. Expose a table-level UI on top of the existing file/Parquet preview

Goals

1. Iceberg table detection

Support recognizing an Iceberg table when the user browses an S3 “folder” that contains:

• A metadata/ directory with one or more v*.metadata.json files
• Possibly a snap-* or metadata/v*.metadata.json pattern as per Iceberg layout

Behavior:

• When listing objects under a prefix (e.g. s3://bucket/warehouse/db/table/), detect if this looks like an Iceberg table.
• Mark such folders in the UI with an “Iceberg table” icon/badge.
• Allow clicking into the table to open an Iceberg Table View instead of a plain folder view.

Detection heuristics (initial):

• Presence of metadata/*.metadata.json files
• JSON structure matching Iceberg table metadata (e.g. fields like format-version, schemas, current-schema-id, snapshots, current-snapshot-id)

2. Fetch and parse Iceberg metadata

Backend changes:

• Add an endpoint (or extend existing S3 proxy) to:
  • GET /iceberg/table?bucket=...&prefix=...
    Returns a normalized JSON object describing:
    • formatVersion
    • schema (current schema, with field IDs, names, types)
    • partitionSpecs
    • snapshots (id, timestamp, parentId, manifestListLocation, summary)
    • currentSnapshotId
• Implement parsing of:
  • metadata.json (JSON)
  • Manifest list (Avro) → list of manifest entries with:
    • manifest_path
    • partition_spec_id
    • added_files_count, existing_files_count, deleted_files_count
    • partition summaries (min/max per partition field)
  • Manifest files (Avro) → list of data/delete file entries with:
    • file path
    • content type (DATA / DELETES)
    • file format (PARQUET, etc.)
    • row count, file size
    • partition values
    • column stats (min/max/nullCount per field ID)

We can use an existing JS/TS library for Iceberg metadata (e.g. icebird from the same authors as hyparquet) or implement minimal parsing ourselves based on the spec.

3. Frontend: Iceberg Table View

New view/route:
/s3/:bucket/*prefix when detected as Iceberg table → Iceberg Table View.

Proposed UI sections:

a) Table overview

• Table path (s3://bucket/warehouse/db/table)
• Format version
• Current schema (table view):
  • Field ID, name, type, nullable
  • Optional: show historical schemas via a dropdown

b) Snapshots

• List of snapshots (from metadata.json):
  • Snapshot ID
  • Timestamp
  • Parent snapshot ID
  • Summary (operation: append, overwrite, etc.)
• Allow selecting a snapshot:
  • When selected, load that snapshot’s manifest list and manifests.
  • Show number of data files, total rows, total size (computed from manifest entries).

Later: support “time travel” by selecting an older snapshot and previewing data as of that snapshot.

c) Partitions (if partitioned table)

• Derive partition fields from partitionSpecs.
• Show a simple partition browser:
  • Distinct partition values (computed from manifest partition summaries or by scanning manifests if needed).
  • Allow filtering by partition values (e.g. date = '2025-01-01').
• When a partition filter is active, show only matching data files.

d) Data files

• List of Parquet data files belonging to the selected snapshot (and optional partition filter).
• For each file:
  • Path
  • Row count
  • Size
  • Partition values
  • Optional: column-level min/max for key columns (from manifest stats)
• Clicking a file opens our existing Parquet Preview modal/page:
  • Schema (from Parquet, but we can also show mapping to Iceberg field IDs)
  • Data preview (first N rows)
  • Optional: highlight columns used in partitioning or filters

This reuses the current Parquet preview component with no changes to the core parsing logic.

4. Minimal MVP scope

For the first iteration, we can limit scope to:

• Detect Iceberg tables in S3
• Load and display:
  • Current schema
  • List of snapshots (basic info)
  • Data files for the current snapshot only
• Show data files in a table with:
  • Path, rows, size, partition values
• Clicking a data file opens the existing Parquet preview

Manifest list/manifest parsing can be done via an existing library (preferred) or a minimal Avro reader focused only on Iceberg’s known schema.

Technical notes

Backend

• Add a new service/module: iceberg/
  • detectIcebergTable(bucket, prefix)
  • getTableMetadata(bucket, prefix) → normalized JSON
  • getSnapshotFiles(bucket, prefix, snapshotId, filters?) → list of data files + stats
• Dependencies:
  • Avro reader for manifest list/manifests (or use icebird if it provides Node-compatible APIs)
  • S3 client already in use

Frontend

• New route: /s3/:bucket/* can branch into:
  • Regular folder view (existing)
  • Iceberg table view (new component IcebergTableView.svelte)
• Components:
  • IcebergTableOverview.svelte – schema, format version
  • IcebergSnapshots.svelte – snapshot list + selection
  • IcebergDataFiles.svelte – data file list, filters, open Parquet preview
• Reuse:
  • Existing ParquetPreview.svelte component for data file preview

Acceptance criteria

• When browsing an S3 prefix that contains an Iceberg table, the UI indicates it’s an Iceberg table.
• Clicking into the table shows:
  • Current schema
  • List of snapshots (at least IDs + timestamps)
  • Data files for the current snapshot
• Each data file shows:
  • Path, row count, size, partition values
• Clicking a data file opens the existing Parquet preview and displays data correctly.
• No changes required to existing Parquet parsing logic; Iceberg support is purely an additional metadata layer.

References

• Iceberg spec: https://iceberg.apache.org/spec/
• Metadata architecture: https://iceberglakehouse.com/apache-iceberg-architecture/
• Manifest list details: https://iceberglakehouse.com/iceberg/iceberg-manifest-list/
• Related JS libraries: hyparquet (Parquet), icebird (Iceberg) from the same authors

Start by reading the existing S3 folder route, S3 proxy, and ParquetPreview.svelte to understand the current browsing and preview flows. Then scope the proposed iceberg/ service and IcebergTableView.svelte components against the MVP acceptance criteria: detect a table, show its current schema and snapshots, list current-snapshot data files, and open existing Parquet previews.