overengineeringstudio / overengineeringstudio/effect-utils

Epic: productionize Notion datasource sync capabilities

Open
#698 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:notion origin:agent type:epic
Dominant language
TypeScript
Stars
82
Forks
2
Avg merge
1d 8h
Merged PRs (30d)
121

Description

Context

PR #683 added the datasource-sync foundation, live E2E harness, demo showcase, NotionMD body adapter integration, safe schema patch subset, and capability documentation.

This epic is the single parent tracker for remaining noninherent limitations that can still be addressed before treating @overeng/notion-datasource-sync as broadly production-ready. It excludes limitations that the public Notion API cannot implement directly.

How To Use This Epic

  • Keep this issue as the parent map and update the child issues for detailed implementation work.
  • Keep closed folded issues in the provenance section only; do not depend on closed issue state for open work.
  • Promote a capability only when it has canonical data models, fake-service regression tests, live Notion evidence when it mutates remote state, and documented fail-closed behavior for unsupported cases.
  • Keep live proof isolated, cleanup-ledger backed, sanitized, and respectful of Notion request limits.

Current Support Snapshot

  • Conservative schema command writes are implemented for add property, rename property, and additive select/multi-select options. Broader destructive/type-changing/status operations remain fail-closed.
  • Public SQLite is close for the current contract: writable rows, read-only changes/conflicts/sync_status, guarded row create/archive/restore, title/description metadata, relation write subsets, and fail-closed private/debug surfaces.
  • Body push uses the NotionMD-backed replace_content shape, but live bidi settlement still needs a focused property-only/body-isolation proof.
  • External URL file attach for empty files properties is supported; durable File Upload lifecycle and broader file replacement/delete/preservation are still open.
  • View inventory is read-only; view writes and query-through-view semantics are still open.
  • Webhook dirty hints exist; templates, comments/discussions, and user/profile privacy boundaries are still open.

Child Workstreams

Release Readiness And Verification
  • #715

Tracks daemon production readiness, large full-replica dry-runs, live mutation soak, cleanup-ledger recovery, serial/rate-limited fixture cleanup, and Notion API rate-limit/backpressure proof.

Public SQLite Contract
  • #716

Tracks status projection, database-ID SQLite VRS alignment, archive/restore ergonomics, conflict resolution lifecycle, terminal CDC retention, choose-local/manual conflict writes, and public/private rebuild semantics.

Live Bidi And Body Safety
  • #717

Tracks property-only sync isolation, clean .nmd settlement, scoped/explainable body conflicts, typed markdown update/restore helpers, and scratch-row-only live tests with snapshot/restore cleanup.

Schema, Properties, Relations, Rollups, And Formulas
  • #718

Tracks safe schema expansion, destructive/type/status migration policy, place, unique_id, wiki verification, people/user identity, unsupported objects, relation checkpoint health, incomplete rollups, formula depth/errors, and inaccessible dependency diagnostics.

Files, Media, Metadata, And Lifecycle
  • #721

Tracks durable File Upload identity, retry/expiry/read-after-write/cleanup, file replacement/delete/preservation, icon/cover media identity, parent/container/trash/lock lifecycle, local-first data-source/database creation, and repair/manifest evolution.

Views And Optional Notion API Surfaces
  • #720

Tracks view create/update/delete authority, query-through-view semantics, API-version gating, templates, comments/discussions, webhook boundaries, and user/profile privacy semantics.

Runtime And Delivery Gates
  • #719

Tracks supported source/runtime CLI paths, fail-closed diagnostics for unsupported Bun/source execution, packaged Node-backed notion sqlite ... guidance, CI/check-all cleanup, oxlint/FOD refresh, and devenv/dt wrapper behavior.

Acceptance Criteria

  • Capability docs distinguish implemented support, guarded/read-only support, feasible follow-up work, and public-API limitations.
  • Production-readiness claims are backed by passing targeted tests plus a green delivery gate or a documented external/tooling blocker.
  • Public SQLite behavior has explicit contracts for pending work, terminal history, unresolved conflicts, unsupported edits, and migration/rebuild semantics.
  • Each child issue either lands its acceptance criteria or documents why the surface remains fail-closed/out of scope.

Folded Context

Closed issue context folded into this epic:

  • #705
  • #706
  • #707
  • #708
  • #709
  • #710
  • #712

Related open umbrellas that remain separate:

  • #16
  • #681
  • #700

Administrative note: #714 was accidentally created through a connector using the wrong GitHub account, then closed as not planned. #715 is the correct child issue for that workstream.

References

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with packages/@overeng/notion-datasource-sync/docs/capabilities.md and docs/vrs/capability-gaps.md, then read the open child issues such as #716, #718, #720, and #721. Choose one scoped workstream before making changes. Done means its acceptance criteria are implemented or the unsupported surface is documented as fail-closed, with targeted tests and required live evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
sqlite, typescript
Domain
api, backend, databases, documentation, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.