apache / apache/gravitino

[Improvement] Support `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` endpoint for server-side scan planning

Open
#11,284 5 comments 0 reactions 1 assignee Claimed by @laserninja View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### What would you like to be improved?

Gravitino Iceberg REST server supports the `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan` endpoint for initiating server-side scan planning, but does not support the `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` endpoint for fetching scan task results.

The Iceberg REST scan planning protocol is a two-step process:

1. **Submit scan plan**: `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan` — returns a plan task token
2. **Fetch scan tasks**: `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` — uses the plan task token to retrieve the actual scan tasks

Gravitino currently advertises the `/plan` endpoint in its `/v1/config` response:
```json
"endpoints": [
...
"POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan",
...
]
```

But it does **not** advertise or implement the `/tasks` endpoint. When a client (e.g., pyiceberg with `scan-planning-mode=server`) successfully submits a scan plan and then attempts to fetch the tasks, it fails with:

```
NotImplementedError: Server does not support endpoint: POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks
```

This makes the server-side scan planning feature incomplete and unusable for clients that follow the two-step protocol.

### Steps to reproduce

1. Configure Gravitino Iceberg REST server with JDBC backend and S3 storage
2. Configure pyiceberg client with `scan-planning-mode=server`
3. Create a table and append data
4. Attempt to scan the table using server-side planning:

```python
from pyiceberg.catalog import load_catalog

catalog = load_catalog(
"local",
**{
"type": "rest",
"uri": "http://localhost:8181/iceberg/",
"s3.endpoint": "http://localhost:9000",
"s3.access-key-id": "admin",
"s3.secret-access-key": "password",
"scan-planning-mode": "server",
"auth": {"type": "noop"},
},
)

table = catalog.load_table("default.my_table")
tasks = list(table.scan().plan_files())
# Raises: NotImplementedError: Server does not support endpoint:
# POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks
```

### Environment

- Gravitino version: main branch (`datastratosandbox/gravitino:1.3.0-SNAPSHOT-20260527024943`)
- Client: pyiceberg (latest)
- Configuration: JDBC catalog backend with S3 storage

### How should we improve?

Implement the `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` endpoint in the Gravitino Iceberg REST server and advertise it in the `/v1/config` response.

The endpoint should accept a plan task token (returned by the `/plan` endpoint) and return the corresponding scan tasks (file scan tasks with data file paths, partition info, etc.).

Alternatively, if Gravitino's scan planning implementation returns all tasks in a single response from the `/plan` endpoint (without requiring a separate `/tasks` call), the `/plan` endpoint response should indicate completion so that clients do not attempt to call `/tasks`.

Reference: [Apache Iceberg REST Catalog Spec - Scan Planning](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.