[Improvement] Support `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` endpoint for server-side scan planning
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### What would you like to be improved?
Gravitino Iceberg REST server supports the `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan` endpoint for initiating server-side scan planning, but does not support the `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` endpoint for fetching scan task results.
The Iceberg REST scan planning protocol is a two-step process:
1. **Submit scan plan**: `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan` — returns a plan task token
2. **Fetch scan tasks**: `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` — uses the plan task token to retrieve the actual scan tasks
Gravitino currently advertises the `/plan` endpoint in its `/v1/config` response:
```json
"endpoints": [
...
"POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan",
...
]
```
But it does **not** advertise or implement the `/tasks` endpoint. When a client (e.g., pyiceberg with `scan-planning-mode=server`) successfully submits a scan plan and then attempts to fetch the tasks, it fails with:
```
NotImplementedError: Server does not support endpoint: POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks
```
This makes the server-side scan planning feature incomplete and unusable for clients that follow the two-step protocol.
### Steps to reproduce
1. Configure Gravitino Iceberg REST server with JDBC backend and S3 storage
2. Configure pyiceberg client with `scan-planning-mode=server`
3. Create a table and append data
4. Attempt to scan the table using server-side planning:
```python
from pyiceberg.catalog import load_catalog
catalog = load_catalog(
"local",
**{
"type": "rest",
"uri": "http://localhost:8181/iceberg/",
"s3.endpoint": "http://localhost:9000",
"s3.access-key-id": "admin",
"s3.secret-access-key": "password",
"scan-planning-mode": "server",
"auth": {"type": "noop"},
},
)
table = catalog.load_table("default.my_table")
tasks = list(table.scan().plan_files())
# Raises: NotImplementedError: Server does not support endpoint:
# POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks
```
### Environment
- Gravitino version: main branch (`datastratosandbox/gravitino:1.3.0-SNAPSHOT-20260527024943`)
- Client: pyiceberg (latest)
- Configuration: JDBC catalog backend with S3 storage
### How should we improve?
Implement the `POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` endpoint in the Gravitino Iceberg REST server and advertise it in the `/v1/config` response.
The endpoint should accept a plan task token (returned by the `/plan` endpoint) and return the corresponding scan tasks (file scan tasks with data file paths, partition info, etc.).
Alternatively, if Gravitino's scan planning implementation returns all tasks in a single response from the `/plan` endpoint (without requiring a separate `/tasks` call), the `/plan` endpoint response should indicate completion so that clients do not attempt to call `/tasks`.
Reference: [Apache Iceberg REST Catalog Spec - Scan Planning](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml)
Contributor guide
Assessment
This issue has not been assessed yet.