cloudflare / cloudflare/workerd

[D1] Raise SQLITE_LIMIT_SQL_LENGTH

Open
#3,049 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
8.7k
Forks
739
Avg merge
2d 20h
Merged PRs (30d)
174

Description

I'm trying to migrate a sqlite DB to D1, which contains string values up to around 500kb. I originally thought this would work in D1, because it appears D1 can store strings up to around 2MB given the `SQLITE_LIMIT_LENGTH`. However when trying to import the table by running `npx wrangler d1 execute database --remote --file=backup.sql` I'm running into:

```
✘ [ERROR] statement too long: SQLITE_TOOBIG
```

And even after getting the file down to a single insert of the form:

```
INSERT INTO data VALUES('<200kb string>')
```

I'm getting the error preventing me from loading this table into D1.

Could the `SQLITE_LIMIT_SQL_LENGTH` limit be raised to match `SQLITE_LIMIT_LENGTH` so that we can import values as large as those that can be stored? Or am I missing something here and there is a way I can get this row imported into D1?

cc: @Frederik-Baetens who recently increased a related limit in https://github.com/cloudflare/workerd/pull/2682

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.