nodejs / nodejs/node

zlib: one-shot gzip()/deflate() results accumulate in arrayBuffers until OOM — GC never prompted (regression in v24.15.0)

Open
#65,600 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
122k
Forks
37.3k
Avg merge
4d 2h
Merged PRs (30d)
283

Description

Version

v24.15.0 through v24.20.0 (first bad release: v24.15.0; still present on v24.19.0 and v24.20.0, the latest 24.x at time of filing). Confirmed clean: v22.12.0, v22.22.0, v24.0.0, v24.13.0, v24.14.0.

Platform
Darwin 25.5.0 arm64

Also reproduced in production on Linux x64 (containerized, cgroup v2) — that is where we hit OOMKills.

Subsystem

zlib

What steps will reproduce the bug?

Call the one-shot convenience API (zlib.gzip()zlib.deflate() behaves identically) in a loop and watch process.memoryUsage().arrayBuffers. No dependencies — save as repro.js:

"use strict";
// One-shot zlib.gzip() in a loop; watch process.memoryUsage().arrayBuffers.
// Run:            node repro.js
// With forced GC: FORCE_GC=1 node --expose-gc repro.js
const zlib = require("node:zlib");

const N = 2000;
// ~120 KB compressible payload (repeating structured JSON)
const chunk = Buffer.from(
  JSON.stringify({ resourceSpans: [{ scopeSpans: [{ spans: [{ name: "POST /x", attrs: { a: 1 } }] }] }] }),
);
const payload = Buffer.concat(Array(Math.ceil((120 * 1024) / chunk.length)).fill(chunk));

const mb = (x) => (x / 1048576).toFixed(1);
function sample(label) {
  if (process.env.FORCE_GC === "1" && global.gc) global.gc();
  const m = process.memoryUsage();
  console.log(
    `${label}\theapUsed=${mb(m.heapUsed)}\texternal=${mb(m.external)}\tarrayBuffers=${mb(m.arrayBuffers)}`,
  );
}

console.log(`node=${process.version} force_gc=${process.env.FORCE_GC === "1"}`);
sample("start");
let i = 0;
(function loop() {
  if (i >= N) return setTimeout(() => sample("end"), 200);
  i++;
  if (i % 500 === 0) sample(`i=${i}`);
  zlib.gzip(payload, (err) => {
    if (err) throw err;
    loop();
  });
})();
How often does it reproduce? Is there a required condition?

Every run, immediately. Two conditions:

  1. The one-shot API (zlib.gzip, zlib.deflate, …). The stream classes are NOT affected — a loop creating a createGunzip() stream per message stays flat, presumably because Close() frees native state deterministically.
  2. A JS heap comfortable enough that V8 does not run major GCs on its own. That is exactly the production steady state of a typical server: our pods sat at ~150 MiB heapUsed against a 432 MiB old-space limit, so nothing ever collected the dead results.
What is the expected behavior? Why is that the expected behavior?

The memory retained by completed one-shot calls should count as GC pressure, so V8 collects the dead result buffers before they accumulate meaningfully. That is what every release up to and including v24.14.0 does — the same loop is flat (sawtooths back down without any forced GC):

=== v24.14.0 ===
node=v24.14.0 force_gc=false
start	heapUsed=3.9	external=1.5	arrayBuffers=0.1
i=500	heapUsed=4.2	external=1.7	arrayBuffers=0.4
i=1000	heapUsed=4.4	external=2.1	arrayBuffers=0.7
i=1500	heapUsed=4.5	external=2.2	arrayBuffers=0.8
i=2000	heapUsed=4.2	external=1.7	arrayBuffers=0.3
end	heapUsed=4.3	external=1.7	arrayBuffers=0.3
What do you see instead?

From v24.15.0 on, arrayBuffers grows linearly with the number of calls and never comes back down:

=== v24.20.0 ===
node=v24.20.0 force_gc=false
start	heapUsed=4.2	external=1.9	arrayBuffers=0.3
i=500	heapUsed=5.2	external=12.2	arrayBuffers=10.6
i=1000	heapUsed=6.9	external=22.6	arrayBuffers=21.0
i=1500	heapUsed=7.2	external=33.1	arrayBuffers=31.4
i=2000	heapUsed=8.5	external=43.5	arrayBuffers=41.9
end	heapUsed=8.5	external=43.5	arrayBuffers=41.9

End-state arrayBuffers across versions (same script, 2000 iterations):

version end arrayBuffers
v22.12.0 2.7 MiB ✅
v24.14.0 0.3 MiB ✅
v24.15.0 31.5 MiB ❌
v24.19.0 41.9 MiB ❌
v24.20.0 41.9 MiB ❌

The buffers are not unreclaimable: with FORCE_GC=1 node --expose-gc repro.js every affected version ends at ~0.3 MiB. The memory is ordinary garbage that nothing ever prompts V8 to collect — which is why the growth is unbounded in a long-running server whose heap is otherwise comfortable.

Growth granularity is the compressed output rounded up to the 16 KB default chunkSize: this compressible 120 KB payload leaks ~16 KB/call; a 16 KB random (incompressible) payload leaks ~31 KB/call; 64 KB random leaks ~94 KB/call.

Additional information

Release-level bisect points at v24.15.0, and the only zlib change in that release is #61717 ("src: refactor compression allocation tracking, enable for zstd"), i.e. commits bef661f182, 3c8f700fd7, 94dbb36d4d, e8079a8297. The behavior is consistent with the one-shot path's retained allocations no longer being reported to V8's external-memory GC-pressure accounting after that refactor.

Real-world impact: @grpc/grpc-js runs zlib.gzip(message, cb) once per outgoing message when compression is enabled (compression-filter.js), so any service exporting OpenTelemetry traces over OTLP/gRPC with CompressionAlgorithm.GZIP leaks per export. After upgrading a production service from Node 22.12 to 24.19 with no dependency changes, its pods accumulated arrayBuffers at ~3 MiB/min and were OOMKilled roughly every 90 minutes; heapUsed stayed flat the whole time. Disabling exporter compression fully mitigates it.

Searched for existing reports before filing: the closest hits are the Blob.stream()/CompressionStream RSS leaks (#64105, #63574, #63708), but those leak RSS rather than arrayBuffers, begin at v24.16 or v26, and are not reclaimed the same way — this one starts exactly at v24.15.0, is fully reclaimed by forced GC, and needs only plain zlib.gzip().

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided repro.js with affected and clean Node.js versions, then inspect the one-shot zlib path and the four commits from #61717 listed in the issue. Trace how completed allocations are reported to V8 and verify that the fix restores GC pressure without affecting stream classes; the reproduction should stop accumulating arrayBuffers without forced GC.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.