trinodb / trinodb/trino-python-client
339 regression: HTTP 200 empty-body retry causes severe spooled-result regression and retries valid empty responses
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 439
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
Expected behavior
The HTTP 200 empty-response retry added for statement responses in f8b1203 / #596 should apply only to Trino statement-protocol responses.
Binary spooled-result downloads and other HTTP operations which legitimately return an empty response body should not be inspected as text or retried solely because their body is empty.
Actual behavior
Since trino-python-client 0.339.0, the HTTP 200 empty-body retry condition is installed on the shared _get, _post, _delete, and _head request wrappers:
lambda response: getattr(response, "status_code", None) == 200
and not getattr(response, "text", "").strip()
Those wrappers are also used by spooled-result requests.
This causes at least two observable regressions.
1. Binary spooled-result downloads become extremely slow
Accessing response.text on a compressed binary segment causes requests to determine an apparent encoding when no usable charset is present.
For compressed spooled segments this can invoke charset_normalizer over the complete binary response, even though the decoded text is immediately discarded.
Measured timings:
| Segment size | 0.338.0 | 0.339.0 | Patched |
|---|---|---|---|
| 0.25 MiB | 1.3 ms | 898.6 ms | 1.3 ms |
| 1.00 MiB | 2.3 ms | 3,666.0 ms | 2.6 ms |
| 4.00 MiB | 5.8 ms | 14,797.7 ms | 6.3 ms |
| 8.00 MiB | 8.6 ms | 29,130.0 ms | 11.3 ms |
The cost is approximately linear with segment size.
Production symptoms included:
DEBUG charset_normalizer: Encoding detection: Unable to determine any suitable charset.
and a query taking approximately 22.2 seconds when its results were distributed over four spooled segments compared with approximately 6.7 seconds for a single segment.
2. Successful spooled-result acknowledgements are retried
Spooled acknowledgement requests return HTTP 200 with an empty body by design.
The retry predicate therefore treats a successful acknowledgement as a transient failure. With the default retry configuration the acknowledgement is sent three times and eventually produces:
INFO trino.client: failed after 3 attempts
despite the first request having succeeded.
This also increases acknowledgement traffic and occupies the acknowledgement worker pool with unnecessary retry backoff.
Additional affected operation
The same empty-response condition is also applied to _head.
A successful HEAD request normally has no response body, so an HTTP 200 heartbeat response can also satisfy this retry condition despite being successful.
Regression
The behaviour was introduced by commit f8b1203 ("Retry on HTTP 200 with empty response body"), which fixed #596.
0.338.0 does not exhibit the spooling regressions.
A local reproduction gives:
| Version | Acknowledgement sent once | 4 MiB fetch | #596 empty-statement-response retry |
|---|---|---|---|
| 0.338.0 | PASS | PASS | FAIL |
| 0.339.0 | FAIL | FAIL | PASS |
| 0.339.0 + proposed patch | PASS | PASS | PASS |
Proposed fix
Scope the HTTP 200 empty-body retry condition to statement-protocol requests rather than the shared HTTP wrappers.
For example:
_get,_post,_delete, and_headretain the generic retry conditions for 429/502/503/504.- Statement GET and POST use statement-specific wrapped request functions which additionally retry an HTTP 200 empty body.
- Test emptiness using
response.content, rather thanresponse.text, so determining whether the body is empty cannot trigger charset detection.
This preserves the fix for #596 while preventing the statement-specific condition from affecting spooled binary downloads, acknowledgements, HEAD requests, or other transport operations.
Workaround
Pin:
trino==0.338.0
This avoids the regression, at the cost of not including the HTTP 200 empty-statement-response retry introduced for #596.
Trino Python client version
0.339.0
Are you willing to submit PR?
Yes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the shared _get, _post, _delete, and _head request wrappers, then trace the statement GET/POST and spooled-result request paths. Confirm the behavior for empty statement responses, binary spooled segments, acknowledgements, and HEAD requests; done means statement retries remain while other successful empty responses are not retried.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100