apache / apache/iceberg

Tracking: Glue Iceberg REST catalog returns HTTP 400 with x-amzn-ErrorType: ThrottlingException instead of 429 for throttling

Open
#17,824 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.10.0 (also affects `main`)

### Query engine

None (direct REST client)

### Please describe the bug 🐞

This tracks a server-side deviation in the AWS Glue Iceberg REST catalog endpoint, plus the
client-side workaround for it in this repo (PR #17710). The endpoint is an AWS-operated service, so
the actual fix (returning 429) has to happen inside AWS. This issue exists to document the behavior
and anchor the workaround.

The AWS Glue Iceberg REST catalog endpoint responds to throttling with **HTTP 400** and an
`x-amzn-ErrorType: ThrottlingException` header, rather than the **HTTP 429** the Iceberg REST
spec mandates for rate limiting.

The REST spec models rate limiting as 429 (see the `loadTable` responses in `open-api/rest-catalog-open-api.yaml`),
and Iceberg's own retry logic keys off the status code. `ExponentialHttpRequestRetryStrategy` retries
429 (and 503/502/504) but treats 400 as a non-retryable client error. So under Glue throttling, clients
see a hard failure with no backoff instead of retrying, even though the condition is transient.

#### Reproduction

SigV4-signed GET requests (service `glue`, region `us-east-2`) to the Glue IRC endpoint, first
resolving the prefix from `/v1/config`, then hammering `loadTable` from 32 threads:

```
GET https://glue.us-east-2.amazonaws.com/iceberg/v1/catalogs//namespaces//tables/
```

Out of ~1920 successful loads, at least one came back throttled:

```
HTTP/1.1 400
Date: Wed, 26 Aug 2026 13:07:44 GMT
Content-Type: application/json
Content-Length: 27
x-amzn-RequestId: a60d6960-0776-46fd-ba82-f5113fe71fb1
x-amzn-ErrorType: ThrottlingException:http://internal.amazon.com/coral/com.amazon.coral.availability/

{"message":"Rate exceeded"}
```

The status line is `400`, the error type is carried only in the `x-amzn-ErrorType` header, and the
body is the AWS Coral `{"message":"Rate exceeded"}` shape rather than the Iceberg `ErrorResponse` schema.

#### Impact

- Generic Iceberg REST clients get a non-retryable 400 on a transient throttle. Instead of backing
off and retrying, they fail the request (or fail the whole commit), which is exactly the opposite
of the intended throttling behavior.
- Any HTTP-status-based retry strategy (Iceberg's default `ExponentialHttpRequestRetryStrategy`, and
most third-party ones) will not retry, because 400 is a client error.

#### Expected behavior

The Glue IRC endpoint should return **429 Too Many Requests** for throttling, per the Iceberg REST
spec, so standard clients back off and retry. The fix is server-side within AWS Glue; there is no
public repo that owns it, so this issue tracks the deviation and the client workaround.

#### Related

- PR https://github.com/apache/iceberg/pull/17710 adds the client-side workaround: retry REST
requests rejected with a 400 that carries the AWS throttling error type. That workaround is only
needed because of this server-side deviation from the spec.

Contributor guide

Open the contributing guide

Research direction

Start with open-api/rest-catalog-open-api.yaml and ExponentialHttpRequestRetryStrategy, then review PR #17710 for the client-side workaround. There is no repository-owned server fix; the issue is complete when the AWS Glue behavior is corrected to return 429 or the workaround remains documented and tracked.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java, openapi
Domain
api, backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.