nodejs / nodejs/node

NODE_V8_COVERAGE can run OOM when compiling a lot of scripts in one process

未关闭
#44,364 6 条评论 0 个 reaction 已指派 1 人 在 GitHub 查看

@joyeecheung 已经在做这个了。

开始于 2022年8月23日。

coverage
主要语言
JavaScript
星标
122k
派生
37.4k
平均合并
4 天 3 小时
30 天内合并 PR
272

描述

For more context, see https://bugs.chromium.org/p/v8/issues/detail?id=13183. In summary, in Node.js NODE_V8_COVERAGE is implemented by taking precise coverage, which holds on to all the feedback vectors of every function (including scripts) that has ever been compiled, even after the functions are gone. My understanding that this is by-design for precise coverage - by the time the coverage is collected, it's possible that the functions have already been GC'ed, but V8 still needs to consult the feedback vectors for invocation counts that were recorded when the functions were still alive. As an alternative V8 provides best-effort coverage that does not hold on to feedback vectors but as a result the data from functions that have been GC'ed by the time the coverage is taken can be missing.

Reproduction (h/t @ptomato):

//  Run this with NODE_V8_COVERAGE=/tmp node --max_old_space_size=30 test.mjs
import vm from 'node:vm';
import v8 from 'node:v8';
const setupCode = new vm.Script('Date.now();'.repeat(10000));

for (let count = 1; count < 50000; count++) {
  const context = {};
  vm.createContext(context);
  setupCode.runInContext(context);
  if (count % 100 === 0) {
    console.log(count, v8.getHeapStatistics());
    console.log(process.memoryUsage());
  }
}

I think there are two options to tackle this:

  1. When NODE_V8_COVERAGE is enabled, periodically collect coverage, which allows V8 to GC the feedback vectors of functions that are no longer alive. We either keep that data in memory and write it out when the process is about to exit (which is what we do now), or flush the collected coverage periodically to the directory specified by NODE_V8_COVERAGE (personally I like the latter better, I believe the existing coverage tooling can already merge the data from different files anyway?) We could add another environment variable for users to control the frequency, and they can use Infinity to tell Node.js to go back to the current behavior.
  2. We can also provide an environment variable for the user to specify that they want best-effort coverage instead of precise coverage.

cc @bcoe

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。