meshery / meshery/meshsync

MeshSync: periodic reconciliation to self-heal missed watch events (design)

Open
#581 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
90
Forks
96
PR merge metrics
No merged PRs in 30d

Description

## Summary
Add a periodic reconcile that re-lists watched resources and publishes only genuine drift, to self-heal missed watch events. Today the informer resync period is `0` (pure watch); a watch gap silently drifts MeshSync's snapshot from cluster truth until an external resync is triggered.

## Key decisions
- **A dedicated LIST-vs-Store diff loop, not a non-zero informer resync period** - a client-go resync only replays the local cache and cannot detect events missed entirely (including deletes). The loop diffs a fresh LIST against the informer's `cache.Store`: LIST minus Store = missed add, higher resourceVersion = missed update, **Store minus LIST = missed delete** (reconstruct the tombstone through the existing DeleteFunc path).
- Publishes only drift (reimplements resourceVersion suppression at the reconcile layer) so it never spams Server; steady state = zero publishes.
- User-tunable interval via the MeshSync CRD, **default off**, with an enforced floor; jittered and staggered across GVRs to avoid re-list storms.

## Risks flagged
A partial/failed LIST must abort that GVR's pass for the tick (never diff a truncated list as a mass-delete). Fixes a **pre-existing `h.stores` unguarded-map race** that the loop's concurrent read is the first to expose.

Complementary to JetStream durability (companion blueprint).

## Full design
[docs/design/fd5-periodic-reconciliation.md](https://github.com/meshery/meshsync/blob/master/docs/design/fd5-periodic-reconciliation.md) (added in #576).

Contributor guide

Open the contributing guide

Research direction

Start with docs/design/fd5-periodic-reconciliation.md and review the informer cache.Store and existing DeleteFunc path described in the issue. Trace how the MeshSync CRD would configure the interval, then validate that the design covers drift-only publishing, failed-list aborts, jitter and staggering, and safe concurrent access to h.stores.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
distributed-systems, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.