nuxt / nuxt/content

D1 Dump restore fails with SQLITE_TOOBIG for large content collections

Open
#3,799 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
3.7k
Forks
746
Avg merge
1d 16h
Merged PRs (30d)
3

Description

Environment
  • Operating System: Linux 6.8.0-1044-azure
  • Node Version: v24.14.0
  • Nuxt Version: 4.4.6
  • CLI Version: 3.35.2
  • Nitro Version: 2.13.4
  • Package Manager: bun@1.3.14
  • Builder: vite@7.3.3
  • User Config: app, colorMode, compatibilityDate, content, css, devtools, experimental, i18n, icon, image, modules, routeRules, ui
  • Runtime Modules: @nuxt/ui@4.8.0, @nuxt/content@3.14.0, @nuxt/image@2.0.0, @nuxtjs/i18n@10.4.0, @vueuse/nuxt@14.3.0
  • Build Modules: -
Version

v3.14.0

Reproduction

No minimal reproduction available - issue occurs with ~1000+ markdown files per collection.

Description

When deploying to Cloudflare Workers with D1 (the officially documented deployment
target), the cold start dump restore fails with SQLITE_TOOBIG.

The dump file generated at build time is
a single gzip+base64 encoded blob - not plain SQL INSERT statements. This format
works correctly for WASM SQLite (client-side navigation) but fails
when the D1 cold start handler attempts to restore it because D1 enforces a hard
limit of 100 KB per SQL statement.

This used to work - the project had 3000+ files and worked fine in 2025. Based on
the changelog, the dump format likely changed around v3.11.0 when SQL queries were
wrapped in a transaction (PR #3670), though we haven't pinpointed the exact commit.

Additional context

generateCollectionInsert() in collection.ts correctly splits runtime INSERT
queries into ≤100 KB chunks (MAX_SQL_QUERY_SIZE = 100000, fixed in PR #3717), but
this split logic is not applied to the dump file used for cold start restore

Logs

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in collection.ts with generateCollectionInsert() and its 100 KB splitting logic, then trace how the build-time dump reaches the D1 cold-start restore path. Reproduce with a collection of roughly 1,000 or more markdown files and compare the dump format with the runtime INSERT chunks. Done means a large D1 deployment restores successfully without SQLITE_TOOBIG while preserving WASM SQLite behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
nuxtjs, sqlite, typescript, wasm
Domain
backend, cloud, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.