Deleted documents remain in embeddings_queue on 1.5.9 with automatically_purge=true (reproduces closed #3793)

Summary

#3793 — "Document deletion leaves plain text and embeddings in db" — was closed as completed on 2025-06-30 with #4884 named as the fix. A commenter reported afterwards that it still reproduced on 1.0.15 and the thread went quiet.

It still reproduces on 1.5.9 (current release), on a store created fresh by 1.5.9, with automatically_purge enabled — which is the default for a new store. embeddings_queue is never trimmed: it grows monotonically and retains the document text and embedding of every deleted record.

Reproduction

chromadb==1.5.9, Python 3.12, macOS arm64, PersistentClient.

import hashlib, os, shutil, sqlite3
import chromadb

DB = os.path.abspath("chroma-persist")
if os.path.exists(DB):
    shutil.rmtree(DB)

def embed(texts):
    return [[(hashlib.sha256(t.encode()).digest()[i % 32] / 255.0) for i in range(32)]
            for t in texts]

def wal(label):
    con = sqlite3.connect(os.path.join(DB, "chroma.sqlite3"))
    n = con.execute("SELECT count(*) FROM embeddings_queue").fetchone()[0]
    cfg = con.execute("SELECT * FROM embeddings_queue_config").fetchall()
    raw = open(os.path.join(DB, "chroma.sqlite3"), "rb").read()
    print(f"{label:26} queue={n:<5} subject on disk: {b'Dana Whitfield' in raw}  {cfg}")

SUBJECT = "Dana Whitfield lives at 42 Alder Row, dana.whitfield@example.com"

client = chromadb.PersistentClient(path=DB)
col = client.create_collection("memories")
col.add(ids=["m1", "m2", "m3"],
        documents=[SUBJECT, "Marcus prefers dark mode", "Kafka runs on 9092"],
        embeddings=embed([SUBJECT, "a", "b"]))
wal("after add 3")

col.delete(ids=["m1", "m2"])
wal("after delete 2")

del col, client
chromadb.api.client.SharedSystemClient.clear_system_cache()
client = chromadb.PersistentClient(path=DB)
col = client.get_collection("memories")
wal("after restart")

docs = [f"Filler record number {i}" for i in range(150)]
col.add(ids=[f"f{i}" for i in range(150)], documents=docs, embeddings=embed(docs))
wal("after 150 more writes")

Actual

after add 3                queue=3     subject on disk: True   [(1, '{"automatically_purge":true,...}')]
after delete 2             queue=5     subject on disk: True   [(1, '{"automatically_purge":true,...}')]
after restart              queue=5     subject on disk: True   [(1, '{"automatically_purge":true,...}')]
after 150 more writes      queue=155   subject on disk: True   [(1, '{"automatically_purge":true,...}')]

The queue only grows. Deleting two records adds two rows rather than removing two.

Expected

With automatically_purge true, entries for records that have been deleted and compacted should not remain in embeddings_queue indefinitely.

What the queue holds

seq=1 op=add     m1   128-byte vector   "Dana Whitfield lives at 42 Alder Row…"
seq=2 op=add     m2   128-byte vector   "Marcus prefers dark mode"
seq=3 op=add     m3   128-byte vector   "Kafka runs on 9092"
seq=4 op=delete  m1   (the add row above is not removed)
seq=5 op=delete  m2   (the add row above is not removed)

The embeddings table is pruned correctly and the API is consistent throughout — count() returns the right number and get() does not return the deleted ids. The residue is only visible below the API.

Other things tried

• chroma vacuum --path <dir> --force — reclaims some space (16 KiB, 4% on the 155-row store above) but does not purge the queue: embeddings_queue stays at 155 rows and the deleted records' text is still in the file afterwards.
• No client or collection method in 1.5.9 exposes a purge, compact or truncate operation.
• Two restarts and 150 further writes, in case a sync threshold gates it. No change.

Why it matters

Unbounded growth. This was the original reporter's complaint on #3793: a delete-then-reinsert workflow grows chroma.sqlite3 forever.

Deleted personal data persists in plaintext. For anyone using Chroma to hold personal data, delete() removes the record from the API while the document text and its embedding stay in the file. That is a data-retention exposure rather than a correctness bug — the API is telling the truth about its own view, and the data is still on disk. Embeddings are also not reliably non-identifying (Morris et al., Text Embeddings Reveal (Almost) As Much As Text), though here the plaintext is present anyway.

Not tested

• Client/server mode. The earlier commenter on #3793 reported the same behaviour on chromadb/chroma:1.0.15 in Docker, but that is their report, not mine.
• Whether a store that has never had automatically_purge written behaves differently.
• Non-macOS platforms.

Ask

Reopen #3793, or treat this as its continuation — happy to move the detail there instead if that is easier to track.

Start by running the supplied Python reproduction with chromadb 1.5.9 and inspecting chroma-persist/chroma.sqlite3, especially embeddings_queue and embeddings_queue_config. Trace the automatically_purge behavior that handles deletes; done means deleted records' text and embeddings no longer remain indefinitely, while API results and queue cleanup remain correct under the shown restart and subsequent-write checks.