nodejs / nodejs/node

v22.x: main thread self-deadlocks when creating a CpuProfiler under GC pressure — backport request for V8 a6eaf75741 (RecursiveMutex for Logger, crbug 41497149)

Ouverte
#65,140 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Langage dominant
JavaScript
Étoiles
122k
Forks
37.3k
Merge moyen
4 j 2 h
PR mergées (30 j)
283

Description

v22.x backport request: V8 a6eaf75741 ("[logging] Use RecursiveMutex for Logger") — main-thread self-deadlock when a CpuProfiler is created

Version

v22.20.0 (still present in v22.23.1 — see below)

Platform
Linux 7.0.0-22-generic #22-Ubuntu SMP PREEMPT_DYNAMIC x86_64 (Docker node:22.20-bookworm-slim)
Subsystem

deps/v8

What steps will reproduce the bug?

Create a v8::CpuProfiler (e.g. any native addon or embedder calling v8::CpuProfiler::New, or in-process profiling agents — we hit it through @sentry/profiling-node, whose N-API Init creates a profiler at require() time) in a process that already has a significant amount of compiled script code, at a moment when the heap is close to the incremental-marking trigger.

CpuProfiler::NewProfilingScopeV8FileLogger::LogCompiledFunctions() dispatches CodeCreateEvents through the Logger dispatcher, which holds a non-recursive base::Mutex while calling listeners (deps/v8/src/logging/code-events.h). ProfilerListener::CodeCreateEvent initializes script line-ends, which allocates; if that allocation crosses the incremental-marking limit, V8 starts a major GC cycle on the same thread: Heap::StartIncrementalMarkingMarkCompactCollector::StartCompactionIsolate::AllowsCodeCompaction()logger()->allows_code_compaction() → attempts to re-acquire the same mutex → the main thread deadlocks against itself, forever.

This is Chromium bug 41497149, fixed upstream in May 2024 by V8 commit a6eaf75741 ("[logging] Use RecursiveMutex for Logger", https://chromium-review.googlesource.com/c/v8/v8/+/5572952) — the commit message describes exactly this scenario. The fix landed in V8 ≥ 12.7, so Node 23/24 have it; Node 22's V8 12.4 does not, and the file is unchanged in v22.23.1 (deps/v8/src/logging/code-events.h still uses plain base::MutexGuard). Node 20/21 are equally affected but EOL.

The patch is minimal (one file, MutexRecursiveMutex in the Logger dispatcher) and V8 12.4 already ships base::RecursiveMutex, so it should float trivially onto deps/v8 in v22.x.

How often does it reproduce? Is there a required condition?

Non-deterministic by nature: the incremental-marking start has to land inside the LogCompiledFunctions walk. In production (NestJS monolith on a 2-vCPU host, addon loaded early in a large module graph, container cold-boot contention) we measured roughly 1 process start in 3-4 freezing forever. Calm restarts of the same build essentially never hit it, which makes it very painful to diagnose in the field.

What is the expected behavior?

Creating a CPU profiler never deadlocks the process.

What do you see instead?

The process freezes before binding any port, with zero output and ~0 CPU, and never crashes (so process managers keep it "online"). gdb backtrace of the frozen main thread (all other threads idle — same-thread self-deadlock):

#1  pthread_mutex_lock
#2  v8::internal::Isolate::AllowsCodeCompaction()
#3  v8::internal::MarkCompactCollector::StartCompaction(...)
#4  v8::internal::IncrementalMarking::StartMarkingMajor()
#6  v8::internal::Heap::StartIncrementalMarking(...)
#7  v8::internal::PagedSpaceAllocatorPolicy::EnsureAllocation(...)
#8  v8::internal::MainAllocator::AllocateRawSlow(...)
#11 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(...)
#12 v8::internal::String::CalculateLineEnds(...)
#13 v8::internal::Script::InitLineEndsInternal(...)
#14 v8::internal::Script::GetPositionInfo(...)
#16 v8::internal::SourcePosition::InliningStack(...)
#17 v8::internal::ProfilerListener::CodeCreateEvent(...)
#18 v8::internal::ExistingCodeLogger::LogExistingFunction(...)
#19 v8::internal::ExistingCodeLogger::LogCompiledFunctions(...)
#20 v8::internal::ProfilingScope::ProfilingScope(...)
#21 v8::internal::CpuProfiler::EnableLogging()
#22 v8::internal::CpuProfiler::CpuProfiler(...)
#23 v8::CpuProfiler::New(...)
#24 Init(napi_env__*, napi_value__*)  (native addon, at require() time)
#25 napi_module_register_by_symbol(...)
#28 node::binding::DLOpen(...)

strace shows the main thread parked in FUTEX_WAIT_PRIVATE on a contended mutex; every other thread is in its normal idle wait.

Additional information

Workarounds we validated/identified while root-causing this:

  • don't create profilers at startup (we made the Sentry profiling integration opt-in — also reported to Sentry: https://github.com/getsentry/sentry-javascript/issues/23178);
  • --no-compact-code-space sidesteps the re-entry (AllowsCodeCompaction short-circuits before touching the Logger mutex), at the cost of disabling code-space compaction;
  • upgrading to Node ≥ 23 removes the bug.

Node 22 is in maintenance LTS until 2027-04 and this is a silent, unrecoverable hang, which is why a backport seems worth it despite maintenance mode.

Guide de contribution

Ouvrir le guide de contribution

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Piste de recherche

Commencez par deps/v8/src/logging/code-events.h et comparez le dispatcher de Logger avec le commit V8 a6eaf75741. Confirmez que V8 12.4 fournit base::RecursiveMutex, puis vérifiez le chemin de création de CpuProfiler sous pression de GC. Le travail est considéré comme terminé lorsque le backport v22.x empêche le self-deadlock du thread principal lors de la création d’un profiler.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
cpp, node.js
Domaine
backend
Type d'issue
Bug
Difficulté
3/5
Temps estimé
1-2 jours
Activité
Calme
Clarté
Clairement spécifiée
Accessibilité débutants
72/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.