GoogleCloudPlatform / GoogleCloudPlatform/cloud-sql-nodejs-connector

Connector.close() + pool.end() leaks TLS sockets for in-flight queries

Đang mở
#562 5 bình luận 2 reaction 0 người được giao Xem trên GitHub
priority: p1 type: bug
Ngôn ngữ chính
TypeScript
Star
97
Fork
16
Merge trung bình
4 ngày 16 giờ
Pull request đã merge (30 ngày)
4

Mô tả

## Bug Description

`Connector.close()` followed by `pool.end()` leaves TLS sockets alive for any mysql2 pool connection that was mid-query when shutdown began. The leak is 1:1 with the number of in-flight queries: with 5 concurrent `SELECT SLEEP(2)` queries running, `whyIsNodeRunning()` 500 ms after shutdown reports 5 `TLSWRAP` handles still pinning the event loop. Their stacks root at:

```
@google-cloud/cloud-sql-connector/dist/mjs/socket.js:107 — tls.connect(socketOpts)
@google-cloud/cloud-sql-connector/dist/mjs/connector.js:148 — getSocket(...)
mysql2/lib/base/connection.js:64 — this.stream = opts.config.stream(opts)
```

Once the socket count grows past a few (e.g. a serving process handling concurrent traffic at the moment `SIGTERM` arrives), the process can fail to exit on graceful shutdown, relying on a forced kill.

Reproduces on both `@google-cloud/cloud-sql-connector@1.9.2` and `@google-cloud/cloud-sql-connector@1.10.0` (latest as of filing).

Suspected cause: `CloudSQLInstance.close()` destroys sockets in `this.sockets`, but pool connections that are currently checked out (mid-query) don't appear to be tracked there. Additionally, `Connector.close()` is synchronous but internally fire-and-forget (`instance.promise.then(inst => inst.close())`), so caller code that immediately follows `close()` with `pool.end()` races the microtask that actually destroys instance sockets.

## Example code (or command)

```js
import mysql from 'mysql2/promise';
import { AuthTypes, Connector } from '@google-cloud/cloud-sql-connector';
import whyIsNodeRunning from 'why-is-node-running';

const INSTANCE_CONNECTION_NAME = 'TODO: your-project:region:instance';
const DATABASE = 'TODO: your-database';
const USER = 'TODO: your-iam-user';

async function main() {
const connector = new Connector();
const clientOpts = await connector.getOptions({
instanceConnectionName: INSTANCE_CONNECTION_NAME,
authType: AuthTypes.IAM,
});
const pool = mysql.createPool({
...clientOpts,
user: USER,
database: DATABASE,
});

// Fire concurrent in-flight queries without awaiting — simulates request
// handlers still running when shutdown begins.
const queries = Array.from({ length: 5 }, (_, i) =>
pool.query('SELECT SLEEP(2), ?', [i]).catch(() => {})
);

// Let the queries actually open TLS sockets.
await new Promise((r) => setTimeout(r, 100));

await pool.end();
connector.close();

// Let libuv settle (collect deactivated timers, flush pending writes).
await new Promise((r) => setTimeout(r, 500));

whyIsNodeRunning();

await Promise.allSettled(queries);
}

main();
```

## Stacktrace

Output of `whyIsNodeRunning()` 500 ms after `pool.end()` + `connector.close()`:

```
% node temp.direct.mjs

...

# TLSWRAP
node_modules/.pnpm/@google-cloud+cloud-sql-connector@1.10.0/node_modules/@google-cloud/cloud-sql-connector/dist/mjs/socket.js:107 - const tlsSocket = tls.connect(socketOpts);
node_modules/.pnpm/@google-cloud+cloud-sql-connector@1.10.0/node_modules/@google-cloud/cloud-sql-connector/dist/mjs/connector.js:148 - const tlsSocket = getSocket({
node_modules/.pnpm/mysql2@3.16.3/node_modules/mysql2/lib/base/connection.js:64 - this.stream = opts.config.stream(opts);
node_modules/.pnpm/mysql2@3.16.3/node_modules/mysql2/lib/base/pool_connection.js:7 - super(options);
node_modules/.pnpm/mysql2@3.16.3/node_modules/mysql2/lib/pool_connection.js:5 - class PoolConnection extends BasePoolConnection {

...

# FILEHANDLE
(unknown stack trace)

# TTYWRAP
temp.direct.mjs:40 - console.log('shutting down while queries are in flight...');

# Timeout
temp.direct.mjs:46 - await new Promise((r) => setTimeout(r, 500));
temp.direct.mjs:46 - await new Promise((r) => setTimeout(r, 500));
```

## How to reproduce

1. `npm init -y && npm pkg set type=module`
2. `npm i @google-cloud/cloud-sql-connector@1.10.0 mysql2@3.16.3 why-is-node-running`
3. Save the example code above as `repro.mjs`, filling in `INSTANCE_CONNECTION_NAME`, `DATABASE`, `USER`.
4. Ensure `gcloud auth application-default login` has been run and the IAM user has access to the instance.
5. `node repro.mjs`
6. Observe `whyIsNodeRunning()` reports N `TLSWRAP` handles rooted at `@google-cloud/cloud-sql-connector/dist/mjs/socket.js` — one per in-flight query that hadn't completed when `pool.end()` was called.

#### Environment details

- OS: MacOS 15.7.4 ARM (M2 Pro)
- Node.js version: v20.20.0 and v24.13.0
- pnpm version: 10.22.0
- `@google-cloud/cloud-sql-connector` version: 1.10.0
- `mysql2` version: 3.16.3

#### Steps to reproduce

1. Create a `Connector` and a `mysql2` pool per the package README.
2. Issue several concurrent queries without awaiting them.
3. Before the queries resolve, call `await pool.end()` and `connector.close()`.
4. Wait long enough (≥ 500 ms) for libuv to collect any handles that were legitimately cleared.
5. Call `whyIsNodeRunning()` (or otherwise inspect the event loop).
6. Expected: no `TLSWRAP` handles from the connector. Actual: one leaked `TLSWRAP` per in-flight query at shutdown time.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Bắt đầu bằng cách đọc vòng đời của connector quanh dist/mjs/socket.js:107 và dist/mjs/connector.js:148, sau đó so sánh với mysql2/lib/base/connection.js:64 và các đường dẫn kết nối pool được hiển thị trong stack trace. Tái hiện chuỗi shutdown với các truy vấn SELECT SLEEP(2) chạy đồng thời và xác minh rằng shutdown hoàn tất không để lại các handle TLSWRAP của connector cho những truy vấn đang thực thi.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
mysql, node.js, typescript
Lĩnh vực
backend, databases, networking
Loại issue
Lỗi
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
45/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.