logseq / logseq/db-test

EDN import (MCP upsertNodes / import-edn) permanently blocked by any pre-existing invalid entity

Open
#1,180 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
28
Forks
2
PR merge metrics
No merged PRs in 30d

Description

Search first

The closest existing reports are about how such entities are created (#1009, #958) or about the repair path being unsafe (#1067, #1068). I could not find a report about the consequence below, so I'm filing it separately.

What Happened?

My DB graph contains 6 invalid entities that predate any current write. Because of them, every EDN import into this graph is rejected, permanently:

attempt result
MCP upsertNodes (built-in MCP server, 127.0.0.1:12315), dry-run: true success — misleading
MCP upsertNodes, dry-run: false API Error: The Imported EDN has 6 validation error(s)
@logseq/cli import-edn --api-server-token <token> Server status 500 … The Imported EDN has 6 validation error(s)

The reported entities have nothing to do with the data being written — the payload is a single new block on today's journal page:

{:pages-and-blocks [{:page {:block/uuid #uuid "00000001-2026-0913-0000-000000000000"}
                     :blocks [{:block/title "some new block"}]}]}

The per-entity error:

{:entity {:db/id 498, :db/ident :user.property/code-L7Xu5sek
          :block/uuid #uuid "6a731a5d-efa1-4033-a2ac-429cd0664004", :block/title "Code"}
 :dispatch-key :property
 :errors {:block/parent {"user-property" ["disallowed key"]}}}

The offending entities are user properties that also exist as pages ("hybrids"), each carrying :block/parent → the Library page:

eids 498, 623, 1542, 1801, 2165, 2352
:user.property/ai-YaOD6aEK      :user.property/events-YMN0fMnP  :user.property/status-AxnqvT7i
:user.property/thesis-poHuho-E  :user.property/code-L7Xu5sek    :user.property/obsidian-L2DQ_5cb

attribute usages: obsidian 45, status 40, code 28, thesis 12, ai 2, events 1 (128 total)

Root cause, read from source: logseq.outliner.op/import-edn-datalogseq.db.frontend.sqlite-export/validate-import-txs runs validate-local-db! over the whole DB, not just the imported transaction, before applying the ops. So one invalid entity that predates the write vetoes all subsequent writes into that graph.

Reproduce the Bug

Minimal and deterministic: two copies of the same graph, differing by one datom, run through the app's own write functions headlessly (no desktop app needed). Verified on current master.

  1. Take any DB graph whose validation is clean.

  2. Poison one user property the way real graphs end up with such entities — see #1009 / #958 for the natural path:

    [:db/add <user-property-eid> :block/parent <page-eid>]
    
  3. Validate → Found 1 entity with errors: … :errors {:block/parent {"user-property" ["disallowed key"]}}

  4. Run the app's write path (the same functions the MCP upsertNodes tool calls) → rejected.

Measured:

graph whole-db validation errors before write result
clean 0 apply-ops! -> nil — block written and persisted
+1 poisoned datom 1 {:error "The Imported EDN has 1 validation error(s)"} — nothing written (block count on the target page unchanged)

Scripts (nbb-logseq + the repo's own deps/*/src):

step 2 — poison the graph (one datom)
(ns inject-bad
  (:require [datascript.core :as d]
            [logseq.db.common.sqlite-cli :as sqlite-cli]
            [logseq.outliner.cli :as outliner-cli]))

(def path (first *command-line-args*))
(def conn (apply outliner-cli/init-conn
                 (conj (sqlite-cli/->open-db-args path)
                       {:classpath "" :import-type :cli/db-import})))
(def db @conn)
(def lib (d/q '[:find ?e . :where [?e :block/title "Library"] [?e :block/name]] db))
(d/transact! conn [[:db/add 498 :block/parent lib]])
(js/process.exit 0)
step 4 — the app's write path (rejected) + whole-db validation
(ns app-path-test
  (:require [datascript.core :as d]
            [logseq.api.db-based.tools :as api-tools]
            [logseq.db.common.sqlite-cli :as sqlite-cli]
            [logseq.db.frontend.validate :as db-validate]
            [logseq.outliner.cli :as outliner-cli]
            [logseq.outliner.op :as outliner-op]))

(def graph-path (first *command-line-args*))
(def page-title (second *command-line-args*))

(defn -main []
  (let [conn (apply outliner-cli/init-conn
                    (conj (sqlite-cli/->open-db-args graph-path)
                          {:classpath "" :import-type :cli/db-import}))
        db @conn
        page-e (d/q '[:find ?e . :in $ ?t :where [?e :block/title ?t] [?e :block/name]] db page-title)
        page-uuid (str (:block/uuid (d/entity db page-e)))
        ops [{:operation "add" :entityType "block"
              :data {:title "new block" :page-id page-uuid}}]
        edn (api-tools/build-upsert-nodes-edn db ops)
        n-before (count (:errors (db-validate/validate-local-db! db {:db-name graph-path})))]
    (println "validation errors BEFORE:" n-before)
    (println "apply-ops! ->"
             (pr-str (try (outliner-op/apply-ops! conn [[:batch-import-edn [edn {}]]]
                                                  {:outliner-op :batch-import-edn})
                          (catch :default e {:error (str (ex-message e))}))))
    (let [db2 @conn]
      (println "validation errors AFTER:"
               (count (:errors (db-validate/validate-local-db! db2 {:db-name graph-path}))))
      (println "blocks on the target page:"
               (count (d/q '[:find [?c ...] :in $ ?p :where [?c :block/parent ?p]] db2 page-e))))
    (js/process.exit 0)))

(-main)
Expected Behavior

An EDN import should be rejected only when the imported data is invalid. A pre-existing invalid entity elsewhere in the graph should not veto unrelated writes — or, at minimum, the error should say that the cause is pre-existing data and offer a repair path. Today the only signal is a count, and the graph has no working way out of the state (see Additional Context).

Files

Scripts are inlined above; I can attach a prepared minimal graph or a scrubbed copy of the affected graph if useful.

Browser, Desktop or Mobile Platform Information

Windows 11; Logseq DB 2.0.1 desktop; reproduced again on a non-release desktop build compiled from master @ be800f1711; @logseq/cli 0.4.3.

Additional Context

How graphs get into this state — both still open

  • #1009 — file→DB import where a user tag collides with a built-in property name produces an invalid, undeletable entity:
    {:logseq.property.class/extends {"user-property" ["disallowed key"]}} — same shape as ours, different key.
  • #958 — EDN Import Error when importing tag that was already created by page import.

The repair path is not usable — both still open

  • #1067 — fix-invalid-blocks! repairs are silent; only the errors are logged (at :debug), so a run that modified data reports :errors nil.
  • #1068 — "Validate graph" silently mutates/deletes data; fix defaults to true, doesn't bump updated-at, doesn't refresh the UI.
    So a user in this state cannot fix it from the UI. The CLI path (logseq graph validate --fix) exists only in the OCaml CLI, which is not distributed as part of @logseq/cli.

Closed does not mean fixed — #940 was closed and then reproduced in the comments afterwards (with CLI output Error (graph-validation-failed): Graph invalid. Found 1 entity with errors); #775 was closed as stale; #1153 was closed as "cannot reproduce".

Not a duplicate of #1009 — there the import reports invalid blocks while importing; here the graph already contains them and every later write is rejected.

Divergence worth knowing@logseq/cli's local mode (-g) does not run this gate: import-edn -g and mcp-server -g both write successfully into a graph the app refuses. Its vendored deps are also older, so logseq validate -g reports errors on current graphs that the app itself considers valid (:logseq.property/type ["should be either :map, :default, …"]). The CLI therefore can't be used to test or reproduce this.

Workaround used (after backing the graph up): retract the disallowed :block/parent datom on the 6 entities — 6 datoms, db 611395 → 611389. All 128 attribute usages, backlinks and children preserved; writes work again. That is the same action fix-invalid-blocks! performs for the built-in-property branch, applied manually and logged instead of silently.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with logseq.outliner.op/import-edn-data and logseq.db.frontend.sqlite-export/validate-import-txs, then run the inline clean-versus-poisoned graph reproduction. Trace validate-local-db! and the app write path to determine how pre-existing errors are handled. Done means unrelated valid EDN writes are no longer vetoed, or the failure identifies the pre-existing data and provides a repair path.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
api, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.