overengineeringstudio / overengineeringstudio/effect-utils

Unify Notion sync semantics across datasource, .nmd, and React renderer

Open
#798 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
82
Forks
2
Avg merge
1d 8h
Merged PRs (30d)
121

Description

Follow-up from #775. We merged the large Notion sync/VRS work, but the architecture review found several places where related packages still do not fully share one principled sync core.

Problem

We now have a useful shared property-write core, but the broader sync semantics are still split across multiple package-local engines:

  • @overeng/notion-datasource-sync has local surface convergence for SQLite + .nmd properties and a dormant body convergence rail.
  • @overeng/notion-md has its own body reconcile engine and standalone property proof provider.
  • @overeng/notion-react has its own cache/diff/drift model, hash discipline, and mutation semantics.

This risks parallel models for desired/observed/base state, conflict classification, digest spaces, drift recovery, and tests.

Findings to address

  1. Property-write core is shared, but provider evidence is incomplete.

    • notion-md feeds placeholder { _tag: "empty" } desired values for some writable types, which can bypass tag-fit because empty fits any property type.
    • notion-datasource-sync currently derives propertyType from the desired value, intentionally making tag-fit a legacy-preserving no-op.
    • Follow-up: thread the real observed Notion property type through datasource planner snapshots and make notion-md construct real canonical desired values through shared codecs.
  2. Body convergence is staged, not complete unification.

    • Datasource-sync calls the convergence engine for .nmd body as a single-surface no-op anchor and leaves remote semantics in the body adapter.
    • The body-body-delegated rail is documented as dormant and currently lacks a user resolution path if activated.
    • Follow-up: before a second body surface lands, add resolver support and make body convergence outcomes observable/testable in the production path.
  3. Notion React renderer still has independent sync semantics.

    • It owns its own cache schema, stable stringify/djb2-style hash, diff planner, drift recovery, checkpointing, and mutation behavior.
    • Follow-up: decide whether React renderer should adopt a shared pure sync contract or explicitly remain a single-writer renderer adapter with documented divergence from datasource/NotionMD sync semantics.
    • Also check stale E2E expectations around cache drift: implementation appears to detect warm top-level drift, while older tests still document no drift detection.
  4. VRS authority needs ratification.

    • Cross-cutting context/notion-db-markdown-sync VRS says it is canonical for the integrated system, but the decision is still proposed.
    • Follow-up: ratify or revise that authority model, then add a package responsibility matrix.

Proposed direction

Do not immediately create another abstraction unless it can replace real duplication. First close the evidence gaps above. Then consider a small pure sync-core package that owns only shared data contracts and decisions, not I/O:

  • SurfaceIdentity
  • SurfaceSnapshot
  • DigestSpace
  • AuthorityMode
  • ReconcileOutcome
  • conflict classification
  • guard/result vocabulary

Adapters would provide evidence and execute commands:

  • datasource-sync: SQLite/control-plane/workspace facts
  • notion-md: .nmd body/property facts
  • notion-react: rendered block-tree desired state and cache/live observations, if it opts into the shared contract

Acceptance criteria

  • Shared property-write core receives real observed property types from every provider that claims tag-fit safety.
  • Body convergence has a production-observable path or is clearly marked as dormant with no claimed unification.
  • React renderer sync semantics are either integrated into the shared contract or explicitly documented as a separate single-writer renderer model.
  • VRS has one ratified canonical hierarchy for integrated Notion sync behavior.
  • Contract tests exercise shared desired/observed/base/conflict behavior across at least two adapters.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the package responsibilities in @overeng/notion-datasource-sync, @overeng/notion-md, and @overeng/notion-react, then review context/notion-db-markdown-sync. Trace the existing property, body, cache, and drift tests before choosing a scope. Done means the acceptance criteria are met, including a ratified VRS authority model and cross-adapter contract tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, developer-experience, tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.