snowflakedb / snowflakedb/snowflake-connector-python

SNOW-4057933: Result polling hangs indefinitely on RemoteDisconnected after query completes server-side

Open
#3,019 2 comments 0 reactions 1 assignee View on GitHub

@sfc-gh-snow-drivers-warsaw-dl is already working on this.

Since Sep 3, 2026.

bug status-triage_done
Dominant language
Python
Stars
730
Forks
574
Avg merge
5h 45m
Merged PRs (30d)
16

Description

Python version

3.10

Operating system and processor architecture

Linux, x86_64 (amd64)

Installed packages
snowflake-connector-python==3.18.0
dbt-snowflake==1.10.2
dbt-core==1.10.15
What did you do?

Ran a long-running query through dbt (a table build that takes a few hours), using key-pair auth, over a network path that goes through a forwarding proxy before reaching Snowflake. That proxy closes any connection that sits quiet for more than 30 seconds.

Sequence observed:

  1. The statement is submitted; the query runs for hours server-side.
  2. The connector polls GET /queries/v1/query-request?requestId=<X> in its result ping-pong loop. Each poll's server-side hold (observed 30-130s) exceeds the proxy's 30s idle timeout, so most polls take one RemoteDisconnected then succeed on the urllib3 retry — noisy but recoverable.
  3. On one poll, the tunnel is closed at the instant the completion response is returned. Those bytes are lost.
  4. From then on the connector re-polls the SAME requestId indefinitely — one RemoteDisconnected('Remote end closed connection without response') roughly every 60-130s, retryCount incrementing — for 5+ hours in one incident.
  5. Snowflake QUERY_HISTORY confirms the query completed successfully shortly after the loop began. The result was simply never collected. The hang only ends when the surrounding process is killed externally.

Log line (repeats unchanged for hours, same requestId):

WARNING:snowflake.connector.vendored.urllib3.connectionpool:Retrying
(Retry(total=0, connect=None, read=None, redirect=None, status=None)) after
connection broken by 'RemoteDisconnected('Remote end closed connection without
response')': /queries/v1/query-request?requestId=<X>&clientStartTime=<...>&retryCount=<n>&retryReason=0&request_guid=<...>
What did you expect to see?

A query that finished successfully server-side should not cause the client to hang forever. it should return the result or raise, rather than re-polling the same requestId unboundedly with no error.

Potential solutions:

  • The result-poll loop honours an overall bound (elapsed/network_timeout) and raises instead of looping indefinitely.
  • On repeated failure for a given requestId, the connector re-fetches the completed result instead of re-issuing the same lost request.
  • At minimum, raise after N consecutive failures on the same requestId so callers can time out and retry, rather than hanging silently.

Distinct from #2030 / #2027 (retry-on-None during polling, already present in 3.18.0): here the connector does keep retrying, the absence of any recovery is the bug. Related client-side report of the same RemoteDisconnected symptom: apache/airflow#50514.

Can you set logging to DEBUG and collect the logs?

Not captured at DEBUG for the incidents

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.