zlib: one-shot gzip()/deflate() results accumulate in arrayBuffers until OOM — GC never prompted (regression in v24.15.0)
Dieses Issue hat noch niemand übernommen.
- Vorherrschende Sprache
- JavaScript
- Sterne
- 122k
- Forks
- 37.3k
- Ø Merge
- 4 T. 2 Std.
- Gemergte PRs (30 T.)
- 283
Beschreibung
Version
v24.15.0 through v24.20.0 (first bad release: v24.15.0; still present on v24.19.0 and v24.20.0, the latest 24.x at time of filing). Confirmed clean: v22.12.0, v22.22.0, v24.0.0, v24.13.0, v24.14.0.
Platform
Darwin 25.5.0 arm64
Also reproduced in production on Linux x64 (containerized, cgroup v2) — that is where we hit OOMKills.
Subsystem
zlib
What steps will reproduce the bug?
Call the one-shot convenience API (zlib.gzip() — zlib.deflate() behaves identically) in a loop and watch process.memoryUsage().arrayBuffers. No dependencies — save as repro.js:
"use strict";
// One-shot zlib.gzip() in a loop; watch process.memoryUsage().arrayBuffers.
// Run: node repro.js
// With forced GC: FORCE_GC=1 node --expose-gc repro.js
const zlib = require("node:zlib");
const N = 2000;
// ~120 KB compressible payload (repeating structured JSON)
const chunk = Buffer.from(
JSON.stringify({ resourceSpans: [{ scopeSpans: [{ spans: [{ name: "POST /x", attrs: { a: 1 } }] }] }] }),
);
const payload = Buffer.concat(Array(Math.ceil((120 * 1024) / chunk.length)).fill(chunk));
const mb = (x) => (x / 1048576).toFixed(1);
function sample(label) {
if (process.env.FORCE_GC === "1" && global.gc) global.gc();
const m = process.memoryUsage();
console.log(
`${label}\theapUsed=${mb(m.heapUsed)}\texternal=${mb(m.external)}\tarrayBuffers=${mb(m.arrayBuffers)}`,
);
}
console.log(`node=${process.version} force_gc=${process.env.FORCE_GC === "1"}`);
sample("start");
let i = 0;
(function loop() {
if (i >= N) return setTimeout(() => sample("end"), 200);
i++;
if (i % 500 === 0) sample(`i=${i}`);
zlib.gzip(payload, (err) => {
if (err) throw err;
loop();
});
})();
How often does it reproduce? Is there a required condition?
Every run, immediately. Two conditions:
- The one-shot API (
zlib.gzip,zlib.deflate, …). The stream classes are NOT affected — a loop creating acreateGunzip()stream per message stays flat, presumably becauseClose()frees native state deterministically. - A JS heap comfortable enough that V8 does not run major GCs on its own. That is exactly the production steady state of a typical server: our pods sat at ~150 MiB heapUsed against a 432 MiB old-space limit, so nothing ever collected the dead results.
What is the expected behavior? Why is that the expected behavior?
The memory retained by completed one-shot calls should count as GC pressure, so V8 collects the dead result buffers before they accumulate meaningfully. That is what every release up to and including v24.14.0 does — the same loop is flat (sawtooths back down without any forced GC):
=== v24.14.0 ===
node=v24.14.0 force_gc=false
start heapUsed=3.9 external=1.5 arrayBuffers=0.1
i=500 heapUsed=4.2 external=1.7 arrayBuffers=0.4
i=1000 heapUsed=4.4 external=2.1 arrayBuffers=0.7
i=1500 heapUsed=4.5 external=2.2 arrayBuffers=0.8
i=2000 heapUsed=4.2 external=1.7 arrayBuffers=0.3
end heapUsed=4.3 external=1.7 arrayBuffers=0.3
What do you see instead?
From v24.15.0 on, arrayBuffers grows linearly with the number of calls and never comes back down:
=== v24.20.0 ===
node=v24.20.0 force_gc=false
start heapUsed=4.2 external=1.9 arrayBuffers=0.3
i=500 heapUsed=5.2 external=12.2 arrayBuffers=10.6
i=1000 heapUsed=6.9 external=22.6 arrayBuffers=21.0
i=1500 heapUsed=7.2 external=33.1 arrayBuffers=31.4
i=2000 heapUsed=8.5 external=43.5 arrayBuffers=41.9
end heapUsed=8.5 external=43.5 arrayBuffers=41.9
End-state arrayBuffers across versions (same script, 2000 iterations):
| version | end arrayBuffers |
|---|---|
| v22.12.0 | 2.7 MiB ✅ |
| v24.14.0 | 0.3 MiB ✅ |
| v24.15.0 | 31.5 MiB ❌ |
| v24.19.0 | 41.9 MiB ❌ |
| v24.20.0 | 41.9 MiB ❌ |
The buffers are not unreclaimable: with FORCE_GC=1 node --expose-gc repro.js every affected version ends at ~0.3 MiB. The memory is ordinary garbage that nothing ever prompts V8 to collect — which is why the growth is unbounded in a long-running server whose heap is otherwise comfortable.
Growth granularity is the compressed output rounded up to the 16 KB default chunkSize: this compressible 120 KB payload leaks ~16 KB/call; a 16 KB random (incompressible) payload leaks ~31 KB/call; 64 KB random leaks ~94 KB/call.
Additional information
Release-level bisect points at v24.15.0, and the only zlib change in that release is #61717 ("src: refactor compression allocation tracking, enable for zstd"), i.e. commits bef661f182, 3c8f700fd7, 94dbb36d4d, e8079a8297. The behavior is consistent with the one-shot path's retained allocations no longer being reported to V8's external-memory GC-pressure accounting after that refactor.
Real-world impact: @grpc/grpc-js runs zlib.gzip(message, cb) once per outgoing message when compression is enabled (compression-filter.js), so any service exporting OpenTelemetry traces over OTLP/gRPC with CompressionAlgorithm.GZIP leaks per export. After upgrading a production service from Node 22.12 to 24.19 with no dependency changes, its pods accumulated arrayBuffers at ~3 MiB/min and were OOMKilled roughly every 90 minutes; heapUsed stayed flat the whole time. Disabling exporter compression fully mitigates it.
Searched for existing reports before filing: the closest hits are the Blob.stream()/CompressionStream RSS leaks (#64105, #63574, #63708), but those leak RSS rather than arrayBuffers, begin at v24.16 or v26, and are not reclaimed the same way — this one starts exactly at v24.15.0, is fully reclaimed by forced GC, and needs only plain zlib.gzip().
Beitragsleitfaden
Erste Schritte
- Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
- Forke das Repository und arbeite in einem Branch.
- Öffne einen Pull Request, der die Issue-Nummer nennt.
Rechercherichtung
Beginne damit, repro.js mit betroffenen und fehlerfreien Node.js-Versionen auszuführen. Untersuche anschließend den One-Shot-zlib-Pfad und die vier in der Issue aufgeführten Commits aus #61717. Verfolge, wie abgeschlossene Allokationen an V8 gemeldet werden, und überprüfe, dass der Fix den GC-Druck wiederherstellt, ohne Stream-Klassen zu beeinträchtigen; die Reproduktion sollte ohne erzwungenen GC keine arrayBuffers mehr ansammeln.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- javascript, node.js
- Bereich
- backend, performance
- Issue-Typ
- Bug
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Aktiv
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 58/100