microsoft / microsoft/mssql-python
Connection pool can briefly exceed max_size when pooling is disabled under load
Nessuno ha ancora preso questa issue.
- Lingua principale
- Python
- Stelle
- 472
- Fork
- 60
- Merge medio
- 2g 11h
- PR unite (30g)
- 36
Descrizione
Describe the bug
The connection pool can transiently exceed its configured max_size when pooling is disabled (or the process begins shutting down) while new physical connections are being opened concurrently. The pool's internal reserved-capacity counter (_current_size) can drift below the true number of live connections, after which the pool opens more physical connections than max_size allows. It is transient and self-corrects as connections close.
Root cause: _current_size is a bare counter with no record of which reservation a given decrement belongs to. The failed-open cleanup runs if (_current_size > 0) --_current_size, while a pool disable/close runs _current_size = 0. If a reset lands between a thread reserving a slot and that same thread's open failing, the thread's decrement cancels a different thread's live reservation instead of its own.
Exception message: none. This is a silent _current_size accounting drift, no exception or stack trace.
To reproduce
This is a rare timing race, so it is not deterministically reproducible from a plain script. It requires a new connection to fail to open at the exact moment pooling is disabled or the process is shutting down. The interleave that produces it, with max_size = 2:
import mssql_python
from concurrent.futures import ThreadPoolExecutor
mssql_python.pooling(max_size=2)
# 1. Thread A calls connect(), reserves a slot -> _current_size = 1,
# then starts opening a new physical connection (outside the pool lock).
# 2. Another thread disables pooling: closePools() sets _current_size = 0.
# 3. Thread B calls connect(), reserves the freed slot -> _current_size = 1.
# 4. Thread A's open fails (handle alloc / network error): its cleanup runs
# --_current_size -> 0, cancelling B's slot instead of A's.
# 5. Thread B's open succeeds, but _current_size now reads 0.
# 6. Further connect() calls see room and open past max_size.
Observable symptom while the count is drifted: more live sessions than the configured cap.
SELECT COUNT(*) FROM sys.dm_exec_sessions WHERE login_name = 'yourapp';
-- transiently returns 3 with max_size = 2
Expected behavior
The number of live physical connections should never exceed max_size, regardless of a connection-open failure racing a pooling disable or process shutdown.
Further technical details
Python version: any
SQL Server version: any
Operating system: any. The race is in the cross-platform C++ pool, not platform specific.
Additional context
Area: mssql_python/pybind/connection/connection_pool.cpp, the Phase 3 failure catch in ConnectionPool::acquire, plus closePools() and ConnectionPool::close.
Regression status: pre-existing for the connect()-failure path. PR #678 moved Connection construction out of _mutex (required to fix the #671 deadlock), which widens the same race to also cover constructor failures such as ODBC env init, handle allocation, and OOM. Raised during review of #678.
Proposed fix: give the pool a generation counter. close()/closePools() bumps it under the lock, a reserver snapshots it at ++_current_size, and the failure cleanup only decrements if the generation still matches. This makes the decrement attributable and closes both the constructor and connect paths. Roughly 6 to 8 lines plus a regression test that forces the interleave.
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Direzione di ricerca
Leggi mssql_python/pybind/connection/connection_pool.cpp, concentrandoti sulla gestione dell'errore della Fase 3 di ConnectionPool::acquire, su closePools() e su ConnectionPool::close. Segui i percorsi di prenotazione e pulizia, quindi esamina come PR #678 ha modificato la costruzione di Connection. Il lavoro è completato quando un test di regressione copre l'interleaving tra errore e disabilitazione e le connessioni fisiche attive non superano mai max_size.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- cpp, python
- Ambito
- backend, databases
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Attiva
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 62/100