Cursor.close() leaks server-side handle for async commands that were never fetched

Open Beginner friendly
#791 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
python, sql
Domain
backend, databases

Research direction

Start in src/databricks/sql/client.py at Cursor.close(), lines 1713-1718, then inspect the backend close_command implementations for Thrift, SEA, and the kernel backend. Reproduce an async submission that never calls get_execution_result(), close the cursor, and verify that the server-side handle is released without breaking active-result cleanup or backend behavior.

Written by the indexing model from the issue text.

Description

bug engineer-bot

Summary

Cursor.close() does not free the server-side statement handle for async-submitted commands when get_execution_result() was never called. The handle leaks until the session closes (where backend-specific sweeps may or may not catch it).

Repro

import databricks.sql

with databricks.sql.connect(...) as conn:
    cur = conn.cursor()
    cur.execute("SELECT * FROM huge_table", async_op=True)
    # User decides not to wait for the result.
    cur.close()
    # Server-side ExecutedAsyncStatement still tracked until conn.close().

Root cause

Cursor.close() at src/databricks/sql/client.py:1713-1718 only nulls active_command_id and closes active_result_set — but for the async-submit-without-fetch case, active_result_set is None, so backend.close_command(active_command_id) is never invoked.

def close(self) -> None:
    """Close cursor"""
    self.open = False
    self.active_command_id = None
    if self.active_result_set:
        self._close_and_clear_active_result_set()

Scope

Affects all three backends: Thrift, SEA, and the new kernel backend. Each one tracks the async handle differently (Thrift via Thrift-server-side state; SEA via the SEA statement; kernel via _async_handles dict), but the shared Cursor.close() path doesn't notify any of them about the unfetched async handle.

Suggested fix

Single change in shared Cursor.close():

def close(self) -> None:
    self.open = False
    if self.active_result_set:
        self._close_and_clear_active_result_set()
    elif self.active_command_id is not None:
        # async submission that never had get_execution_result called —
        # the result-set close path didn't fire, so issue an explicit
        # close_command to free the server-side handle.
        try:
            self.backend.close_command(self.active_command_id)
        except Exception as exc:
            logger.warning("close_command on cursor close failed: %s", exc)
    self.active_command_id = None

Safe across all backends: Thrift's close_command sends TCloseOperation (valid call), SEA's sends a cancel/close, kernel's is already tolerant of unknown ids (no-op).

Surfaced by

Review of PR #787 (kernel backend). Documented as P0 #1 in https://github.com/databricks/databricks-sql-python/pull/787#issuecomment-4475165482 — but the leak is pre-existing on Thrift and SEA too, not specific to the kernel backend.

Dominant language
Python
Stars
233
Forks
152
Avg merge
21h 5m
Merged PRs (30d)
10

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from databricks/databricks-sql-python

All issues in databricks/databricks-sql-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.