nodejs / nodejs/node

zlib: one-shot gzip()/deflate() results accumulate in arrayBuffers until OOM — GC never prompted (regression in v24.15.0)

未关闭
#65,600 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

主要语言
JavaScript
星标
122k
派生
37.4k
平均合并
4 天 3 小时
30 天内合并 PR
272

描述

Version

v24.15.0 through v24.20.0 (first bad release: v24.15.0; still present on v24.19.0 and v24.20.0, the latest 24.x at time of filing). Confirmed clean: v22.12.0, v22.22.0, v24.0.0, v24.13.0, v24.14.0.

Platform
Darwin 25.5.0 arm64

Also reproduced in production on Linux x64 (containerized, cgroup v2) — that is where we hit OOMKills.

Subsystem

zlib

What steps will reproduce the bug?

Call the one-shot convenience API (zlib.gzip()zlib.deflate() behaves identically) in a loop and watch process.memoryUsage().arrayBuffers. No dependencies — save as repro.js:

"use strict";
// One-shot zlib.gzip() in a loop; watch process.memoryUsage().arrayBuffers.
// Run:            node repro.js
// With forced GC: FORCE_GC=1 node --expose-gc repro.js
const zlib = require("node:zlib");

const N = 2000;
// ~120 KB compressible payload (repeating structured JSON)
const chunk = Buffer.from(
  JSON.stringify({ resourceSpans: [{ scopeSpans: [{ spans: [{ name: "POST /x", attrs: { a: 1 } }] }] }] }),
);
const payload = Buffer.concat(Array(Math.ceil((120 * 1024) / chunk.length)).fill(chunk));

const mb = (x) => (x / 1048576).toFixed(1);
function sample(label) {
  if (process.env.FORCE_GC === "1" && global.gc) global.gc();
  const m = process.memoryUsage();
  console.log(
    `${label}\theapUsed=${mb(m.heapUsed)}\texternal=${mb(m.external)}\tarrayBuffers=${mb(m.arrayBuffers)}`,
  );
}

console.log(`node=${process.version} force_gc=${process.env.FORCE_GC === "1"}`);
sample("start");
let i = 0;
(function loop() {
  if (i >= N) return setTimeout(() => sample("end"), 200);
  i++;
  if (i % 500 === 0) sample(`i=${i}`);
  zlib.gzip(payload, (err) => {
    if (err) throw err;
    loop();
  });
})();
How often does it reproduce? Is there a required condition?

Every run, immediately. Two conditions:

  1. The one-shot API (zlib.gzip, zlib.deflate, …). The stream classes are NOT affected — a loop creating a createGunzip() stream per message stays flat, presumably because Close() frees native state deterministically.
  2. A JS heap comfortable enough that V8 does not run major GCs on its own. That is exactly the production steady state of a typical server: our pods sat at ~150 MiB heapUsed against a 432 MiB old-space limit, so nothing ever collected the dead results.
What is the expected behavior? Why is that the expected behavior?

The memory retained by completed one-shot calls should count as GC pressure, so V8 collects the dead result buffers before they accumulate meaningfully. That is what every release up to and including v24.14.0 does — the same loop is flat (sawtooths back down without any forced GC):

=== v24.14.0 ===
node=v24.14.0 force_gc=false
start	heapUsed=3.9	external=1.5	arrayBuffers=0.1
i=500	heapUsed=4.2	external=1.7	arrayBuffers=0.4
i=1000	heapUsed=4.4	external=2.1	arrayBuffers=0.7
i=1500	heapUsed=4.5	external=2.2	arrayBuffers=0.8
i=2000	heapUsed=4.2	external=1.7	arrayBuffers=0.3
end	heapUsed=4.3	external=1.7	arrayBuffers=0.3
What do you see instead?

From v24.15.0 on, arrayBuffers grows linearly with the number of calls and never comes back down:

=== v24.20.0 ===
node=v24.20.0 force_gc=false
start	heapUsed=4.2	external=1.9	arrayBuffers=0.3
i=500	heapUsed=5.2	external=12.2	arrayBuffers=10.6
i=1000	heapUsed=6.9	external=22.6	arrayBuffers=21.0
i=1500	heapUsed=7.2	external=33.1	arrayBuffers=31.4
i=2000	heapUsed=8.5	external=43.5	arrayBuffers=41.9
end	heapUsed=8.5	external=43.5	arrayBuffers=41.9

End-state arrayBuffers across versions (same script, 2000 iterations):

version end arrayBuffers
v22.12.0 2.7 MiB ✅
v24.14.0 0.3 MiB ✅
v24.15.0 31.5 MiB ❌
v24.19.0 41.9 MiB ❌
v24.20.0 41.9 MiB ❌

The buffers are not unreclaimable: with FORCE_GC=1 node --expose-gc repro.js every affected version ends at ~0.3 MiB. The memory is ordinary garbage that nothing ever prompts V8 to collect — which is why the growth is unbounded in a long-running server whose heap is otherwise comfortable.

Growth granularity is the compressed output rounded up to the 16 KB default chunkSize: this compressible 120 KB payload leaks ~16 KB/call; a 16 KB random (incompressible) payload leaks ~31 KB/call; 64 KB random leaks ~94 KB/call.

Additional information

Release-level bisect points at v24.15.0, and the only zlib change in that release is #61717 ("src: refactor compression allocation tracking, enable for zstd"), i.e. commits bef661f182, 3c8f700fd7, 94dbb36d4d, e8079a8297. The behavior is consistent with the one-shot path's retained allocations no longer being reported to V8's external-memory GC-pressure accounting after that refactor.

Real-world impact: @grpc/grpc-js runs zlib.gzip(message, cb) once per outgoing message when compression is enabled (compression-filter.js), so any service exporting OpenTelemetry traces over OTLP/gRPC with CompressionAlgorithm.GZIP leaks per export. After upgrading a production service from Node 22.12 to 24.19 with no dependency changes, its pods accumulated arrayBuffers at ~3 MiB/min and were OOMKilled roughly every 90 minutes; heapUsed stayed flat the whole time. Disabling exporter compression fully mitigates it.

Searched for existing reports before filing: the closest hits are the Blob.stream()/CompressionStream RSS leaks (#64105, #63574, #63708), but those leak RSS rather than arrayBuffers, begin at v24.16 or v26, and are not reclaimed the same way — this one starts exactly at v24.15.0, is fully reclaimed by forced GC, and needs only plain zlib.gzip().

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

首先使用受影响的 Node.js 版本和正常的 Node.js 版本运行 repro.js,然后检查 one-shot zlib 路径以及 issue 中列出的 #61717 的四个 commit。跟踪已完成的分配如何报告给 V8,并验证 fix 在不影响 stream 类的情况下恢复 GC 压力;在不强制 GC 的情况下,复现应停止累积 arrayBuffers。

由索引模型根据 Issue 内容生成。

评估

技术栈
javascript, node.js
领域
backend, performance
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
活跃
描述清晰度
基本清楚
新手友好度
58/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。