duckdb / duckdb/duckdb-node

Memory leak on any query

Open
#55 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
91
Forks
36
PR merge metrics
No merged PRs in 30d

Description

Running any query in duckdb-node leaks memory. Here is a simple test script that creates a connection and then runs a simple `SELECT 42 AS fortytwo` SQL statement every 100ms. The script also publishes memory usage to a file every minute. Memory never stops going up.

```javascript
const duckdb = require("duckdb");
const fs = require("fs");

// Run simple duckdb query on repeat
const db = new duckdb.Database(":memory:");
const con = db.connect();
const stmt = con.prepare("SELECT 42 AS fortytwo");

function test() {
stmt.all();
}
setInterval(test, 100);

// Capture memory stats over time and write to file
const path = "./data.csv";
fs.writeFileSync(path, "rss,heapTotal,heapUsed\n");
const stream = fs.createWriteStream(path, { flags: "a" });
setInterval(() => {
const memory = process.memoryUsage();
const rss = memory.rss / (1024 * 1024);
const heapTotal = memory.heapTotal / (1024 * 1024);
const heapUsed = memory.heapUsed / (1024 * 1024);
console.log(`rss: ${rss} heapTotal: ${heapTotal} heapUsed: ${heapUsed}`);
stream.write(`${rss},${heapTotal},${heapUsed}\n`);
}, 60000);

```

Here is a chart of the resulting memory stats, when run for 50 minutes on an MBP M2, node v18.18.2, duckdb 0.10.0
CleanShot 2024-03-04 at 13 59 58@2x

After 2 hours
CleanShot 2024-03-04 at 15 22 21@2x

EDIT: updated script to remove recursive call

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.