Local Zotero library: new or edited items do not appear until Zotero checkpoints its WAL file

Bug description

The Quarto extension reads the local Zotero library. It copies zotero.sqlite and queries the copy. Zotero writes new and changed items to a -wal file first. Zotero moves this data into the main zotero.sqlite file only at intervals. This process is a checkpoint.

Quarto's staleness check compares only the mtime of the main zotero.sqlite file. As a result, it does not detect newer data in the -wal file.

Because of this, an item that you add or edit in the Zotero desktop app does not appear in the Visual Editor's Insert Citation picker. It appears only after Zotero checkpoints the WAL on its own. Zotero checkpoints on idle time, at a size limit, or when you quit Zotero.

Steps to reproduce

1. Set Quarto > Zotero > Library to local.
2. Make sure that the citation picker works. See #1101.
3. Add a new item to your Zotero library in the Zotero desktop app.
4. In the Visual Editor, open Insert > Citation.
5. Search for the new item.

Result: the new item does not appear. Zotero shows the new item at once in its own library view.

Evidence

$ stat -f "%N  mtime=%Sm" ~/Zotero/zotero.sqlite ~/Zotero/zotero.sqlite-wal
/Users/juliasilge/Zotero/zotero.sqlite      mtime=Sep  9 11:58:55 2026
/Users/juliasilge/Zotero/zotero.sqlite-wal  mtime=Sep  9 12:33:24 2026

The -wal file is 35 minutes newer than the main file. At that time, the new item still did not appear. This shows that the new data was in the WAL file, unread.

Root cause

node-sqlite3-wasm is the library that packages/editor-server/src/core/zotero/local/db.ts uses to read the local database. This library cannot open a WAL-mode database. Direct tests show this, even when the -wal and -shm companion files are next to the main file.

Zotero uses WAL mode by default in current versions.

To work around this, the copy-then-read path in db.ts changes the SQLite file-header bytes that mark WAL mode. This change lets Quarto open the copy (see the fix for #1101).

This workaround reads only the data that Zotero has already checkpointed into the main file. It cannot read data that exists only in the -wal file.

Possible directions

1. Run a passive checkpoint with the system sqlite3 command-line tool. This option runs PRAGMA wal_checkpoint(PASSIVE) on the live database before the copy step, when the -wal file's mtime is newer than the main file's. A passive checkpoint is safe to run while Zotero is open. This option adds a dependency on an external process. The sqlite3 tool is not always on the PATH, especially on Windows.
2. Use a different SQLite library that supports WAL mode. For example, better-sqlite3 supports WAL mode natively. This option removes the need for the header-patch workaround. This is a larger change: it replaces a pure WASM dependency with a native module. A native module needs a prebuilt binary for each platform and architecture.

Related

• I found this issue while I tested the fix for #1101.
• That fix also corrects two other bugs: a race in the Zotero configuration push to the language server, and a concurrency fault where two citation-lookup requests can use the same cached copy of zotero.sqlite at the same time.

Start in packages/editor-server/src/core/zotero/local/db.ts and trace the staleness check and copy-then-read path. Compare the passive sqlite3 checkpoint and WAL-capable library directions described in the issue, then verify that newly added or edited Zotero items appear before a checkpoint. Done means the local citation picker reads current WAL-backed data.