[FixtureCatalog][Dev][Scan Planning] Catalog server returns invalid (cut off) JSON response
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Apache Iceberg version
1.11.0 (latest release)
### Query engine
Other
### Please describe the bug 🐞
Spark sql:
```sql
CREATE TABLE default.alter_partition_drop_column(
a VARCHAR(10),
b BOOLEAN,
c DATE,
d INTEGER
) PARTITIONED BY (a, c);
INSERT INTO default.alter_partition_drop_column VALUES
('test', true, CAST('1983-05-21' AS DATE), 2324);
ALTER TABLE default.alter_partition_drop_column DROP PARTITION FIELD c;
ALTER TABLE default.alter_partition_drop_column DROP COLUMN c;
```
docker-compose image:
`apache/iceberg-rest-fixture:latest`
docker-compose.yml
```yml
services:
rest:
image: apache/iceberg-rest-fixture:latest
container_name: iceberg-rest
networks:
iceberg_net:
ports:
- 8181:8181
environment:
- AWS_ACCESS_KEY_ID=admin
- AWS_SECRET_ACCESS_KEY=password
- AWS_REGION=us-east-1
- CATALOG_WAREHOUSE=s3://warehouse/
- CATALOG_IO__IMPL=org.apache.iceberg.aws.s3.S3FileIO
- CATALOG_S3_ENDPOINT=http://minio:9000
minio:
image: minio/minio
container_name: minio
environment:
- MINIO_ROOT_USER=admin
- MINIO_ROOT_PASSWORD=password
- MINIO_DOMAIN=minio
networks:
iceberg_net:
aliases:
- warehouse.minio
volumes:
- type: bind
source: ../data/generated/iceberg/fixture-latest/
target: /data
ports:
- 9001:9001
- 9000:9000
command: ["server", "/data", "--console-address", ":9001"]
mc:
depends_on:
- minio
image: minio/mc
container_name: mc
networks:
iceberg_net:
environment:
- AWS_ACCESS_KEY_ID=admin
- AWS_SECRET_ACCESS_KEY=password
- AWS_REGION=us-east-1
entrypoint: |
/bin/sh -c "
until /usr/bin/mc alias set minio http://minio:9000 admin password; do
echo 'Waiting for MinIO...';
sleep 2;
done;
# Wait for bucket to exist or create it
if ! /usr/bin/mc ls minio/warehouse > /dev/null 2>&1; then
/usr/bin/mc mb minio/warehouse;
fi
# Optional: clean the namespace for testing
/usr/bin/mc rm -r --force minio/warehouse/default || true;
# Create the `default` namespace (directory) and dummy metadata file
/usr/bin/mc mb minio/warehouse/default;
echo '{}' > /tmp/metadata.json;
/usr/bin/mc cp /tmp/metadata.json minio/warehouse/default/metadata/metadata.json;
# Make it public (optional)
/usr/bin/mc policy set public minio/warehouse;
echo 'MinIO warehouse and default namespace initialized.';
tail -f /dev/null
"
networks:
iceberg_net:
```
Request to `http://127.0.0.1:8181/v1/namespaces/default/tables/alter_partition_drop_column/plan`:
```json
{"snapshot-id":7435989651564931343,"case-sensitive":true,"use-snapshot-schema":false}
```
Response from the server catalog:
```json
{"status":"completed","plan-id":"sync-32f14095-0fed-4bc4-b1ed-aa4a2b4174b7","file-scan-tasks":[{"data-file":{"spec-id":0,"content":"data","file-path":"s3://warehouse/default/alter_partition_drop_column/data/a=test/c=1983-05-21/00000-2-94359d00-02aa-4c14-9f97-48dff44953e2-0-00001.parquet","file-format":"parquet","partition":["test"
```
With headers, to verify the response is correctly received:
```
request_error = ""
reason = "OK"
headers = {
headers = size=5 {
[0] = (first = "Content-Length", second = "332")
[1] = (first = "Content-Type", second = "application/json")
[2] = (first = "Server", second = "Jetty(12.1.8)")
[3] = (first = "Date", second = "Tue, 21 Jul 2026 20:11:24 GMT")
[4] = (first = "Vary", second = "Accept-Encoding")
}
}
success = true
```
It looks like the server hit an error while constructing the result, probably because the field that the data file was partitioned on is no longer part of the schema.
But that error appears to get swallowed, resulting in a 200 response.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
Contributor guide
Research direction
Start with the docker-compose reproduction and send the plan request to /v1/namespaces/default/tables/alter_partition_drop_column/plan. Trace the catalog server's handling of the plan response and the error path that produces the truncated JSON. Done means the response is valid JSON and the underlying failure is surfaced rather than returned with HTTP 200.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker-compose, java, spark
- Domain
- api, backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100