Turbopack: poisoned persistent cache panics all workers on restore ("Every task must have a task type") — deadlocked compiles, ~900% CPU idle spin, OOM crash loop
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 142k
- Forks
- 32.4k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 351
Description
Link to the code that reproduces this issue
https://github.com/shashwat46/turbopack-persistent-cache-panic-repro
To Reproduce
The failure state is a poisoned Turbopack persistent cache (.next/dev/cache/turbopack, 2.1 GB of RocksDB-style .sst files). Restoring it and running next dev reproduces the failure deterministically, every time:
next devboots normally (✓ Ready in 231ms)- within seconds — before any HTTP request — ~10 tokio workers panic:
thread 'tokio-runtime-worker' panicked at turbopack/crates/turbo-tasks-backend/src/backend/operation/mod.rs:966:13:
Every task must have a task type TaskGuard { task_id: TaskId { id: 2147485064 }, storage: TaskStorage {
... persistent_task_type: None,
flags: TaskFlags { ... meta_restored: true, data_restored: true, ... },
lazy: [InProgress(Scheduled { done_event: Event, reason: ActivateInitial })] } }
- any request needing fresh compilation hangs forever (
GET /> 60 s, no response) - under interactive use the process settles into 600–975% CPU while fully idle (verified: no requests for 30+ min, zero fs events in the watched tree). A macOS
sampleshows all tokio workers on-CPU insidenext-swcwith spin-lock yields (cthread_yield/swtch_pri) at the leaves, and worker thread IDs climbing continuously (panic → respawn cycle) - RSS oscillates 1 → 4.8 GB until the server dies (OOM); the
next devwrapper/user restarts it and the cycle repeats. This burned ~2.5 hours across 3 boots before diagnosis.
The linked repo contains a minimal app matching the failing versions, the sanitized boot log, the sample stack analysis, and the invalidation-marker evidence. The poisoned cache itself embeds compiled proprietary application code, so it is retained privately — available to maintainers on request, including running instrumented builds against it.
Suspected corruption origin: torn write during cache persist on hard process death (SIGKILL/OOM — note this bug's own OOM loop makes that recurrent). .sst timestamps place the poisoning in the prior evening's dev session.
Self-healing also failed in the wild: the affected instance wrote .next/dev/cache/turbopack/__turbo_tasks_invalidated_db ("reason_code": "turbo_tasks_backend::database::db_invalidation::invalidation_reasons::PANIC") mid-run — and then kept grinding the poisoned DB at ~900% CPU for 30+ more minutes instead of discarding it; a later boot still restored poisoned tasks with that marker present on disk. To be fair: in controlled replays (marker planted into healthy and poisoned caches, with/without NEXT_TURBOPACK_TRACING), startup honored the marker correctly — the wild non-honoring seems to need an extra condition we could not isolate (in-place DB state vs a fresh copy, or timing around the mid-run marker write).
Current vs. Expected behavior
Current: a restored task with persistent_task_type: None panics the worker pool → deadlocked compiles + idle CPU spin + OOM crash loop, recurring on every restart because the poisoned DB stays on disk.
Expected:
- a restored task without a task type invalidates the cache and falls back to a cold build — never a panic loop
- an instance that writes the PANIC invalidation marker stops trusting the poisoned DB immediately (cold restart of turbo-tasks), rather than continuing to spin against it
Workaround (verified): rm -rf .next/dev/cache/turbopack → panics 10 → 0, GET / from infinite hang → 200 in 4.7 s, idle CPU 975% → 0.2%.
Provide environment information
Operating System:
Platform: darwin Arch: arm64 (M-series, 10 cores, 24 GB)
Version: Darwin 25.5.0
Binaries:
Node: 26.3.0
Relevant Packages:
next: 16.2.6 (Turbopack, default)
react: 19.2.6
react-dom: 19.2.6
typescript: 5.9.3
Canary note: could not verify on canary — the poisoned artifact is version-keyed to 16.2.6, and a synthetic corruption could not be fabricated in the minimal app so far. Release notes for 16.2.7–16.2.11 mention no related fix.
Which area(s) are affected?
Turbopack, Persistent caching
Additional context
Almost certainly the same underlying bug as #93896 (identical symptoms: 16.2.6, M-series, 600%+ CPU; auto-closed without diagnosis). Earlier filings of this diagnosis (#96091, #96092) were auto-closed by the triage bot over the reproduction-link section format — this issue supersedes both.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with turbopack/crates/turbo-tasks-backend/src/backend/operation/mod.rs around line 966, then use the linked turbopack-persistent-cache-panic-repro to reproduce the poisoned-cache restore with next dev. Verify behavior against the reported panic and invalidation-marker cases; done means the poisoned cache is discarded or invalidated, compilation falls back to a cold build, and workers do not enter the panic, hang, or CPU-spin loop.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nextjs
- Domain
- build-system, devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100