ruvnet / ruvnet/RuView

Subject: Multistatic fusion can't hold a 60ms guard on 4x ESP32-S3 — mesh timestamps carry 0.28–1.5s residual error

Open
#1,710 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug firmware hardware
Dominant language
Rust
Stars
94.5k
Forks
12.5k
Avg merge
21h 27m
Merged PRs (30d)
43

Description

Subject: Multistatic fusion can't hold a 60ms guard on 4x ESP32-S3 — mesh timestamps carry 0.28–1.5s residual error

Hi [author],

I'm evaluating your multistatic fusion on 4x ESP32-S3 (devkitc, no-display, DATA capture
enabled, single channel 6) on WINDOWS + a host UDP relay (scripts/udp-relay.py, a pure
byte-forward 5005→5006; the standard workaround for Docker Desktop's multi-source UDP
collapse on Windows). Important: the phenomenon below reproduces identically with a
direct capture (no relay) — the relay is a transparent byte-forwarder and is NOT the
cause. I implemented an ADR-018 extension (Plan A: u64 mesh-aligned timestamp in the CSI
frame at bytes 20..27, byte19 bit5 marks presence) and a host-side robustness fix, but I
cannot hold the 60 ms guard. I've isolated it to three stacked causes.

WHAT I'VE ALREADY DONE / OPTIMIZED

  • Plan A — mesh timestamp stamped in-frame (firmware) + parsed + used to timestamp
    fusion (Rust). Backward compatible; when the mesh is healthy, same-instant mesh
    timestamps agree within ~4–28 ms and fusion is clean at 800 ms guard.
  • Host-side robustness — the fusion frame set EXCLUDES any node whose latest frame is
    not mesh-sync-valid, so a desynced node's host-arrival timestamp can't mix into the
    mesh-aligned set and poison the spread guard (per-node count still falls back).
    Tested: 244 bin + 529 lib green.
  • Correct but INSUFFICIENT — see (1): a node that reports sync-valid but has a biased
    timestamp is still admitted.

Subject: Multistatic fusion can't hold a 60ms guard on 4x ESP32-S3 — mesh timestamps carry 0.28–1.5s residual error

Hi [author],

I'm evaluating your multistatic fusion on 4x ESP32-S3 (devkitc, no-display, DATA capture
enabled, single channel 6) on WINDOWS + a host UDP relay (scripts/udp-relay.py, a pure
byte-forward 5005→5006; the standard workaround for Docker Desktop's multi-source UDP
collapse on Windows). Important: the phenomenon below reproduces identically with a
direct capture (no relay) — the relay is a transparent byte-forwarder and is NOT the
cause. I implemented an ADR-018 extension (Plan A: u64 mesh-aligned timestamp in the CSI
frame at bytes 20..27, byte19 bit5 marks presence) and a host-side robustness fix, but I
cannot hold the 60 ms guard. I've isolated it to three stacked causes.

WHAT I'VE ALREADY DONE / OPTIMIZED

  • Plan A — mesh timestamp stamped in-frame (firmware) + parsed + used to timestamp
    fusion (Rust). Backward compatible; when the mesh is healthy, same-instant mesh
    timestamps agree within ~4–28 ms and fusion is clean at 800 ms guard.
  • Host-side robustness — the fusion frame set EXCLUDES any node whose latest frame is
    not mesh-sync-valid, so a desynced node's host-arrival timestamp can't mix into the
    mesh-aligned set and poison the spread guard (per-node count still falls back).
    Tested: 244 bin + 529 lib green.
  • Correct but INSUFFICIENT — see (1): a node that reports sync-valid but has a biased
    timestamp is still admitted.

ROOT CAUSES (dependency order)

(1) Mesh timebase precision — BLOCKING
Single elected leader (node3), all 4 nodes report sync-valid (byte19 bit4), yet mesh
timestamps are not tightly aligned to the leader epoch:
node2 = leader − 0.28 s
node4 = leader − 0.28 s
node1 = leader − 1.5 s (transiently up to 838 s)
The ESP-NOW offsets (sync packet 0xC511A110) are large but inaccurate at residual
level: node1 reports offset ≈ −122.7 s yet still sits 1.5 s off the leader after
applying it — a ~1.5 s bias that looks like a stale/bad offset baseline, not jitter.
So even perfectly co-sampled frames report timestamps 1.5 s apart and the guard fires
regardless of sampling precision. Firmware-side (c6_sync_espnow EMA / offset baseline).

(2) Sampling simultaneity (passive sniffing)
Each node's CSI fires on whichever PPDU IT sees. The AP's 100 ms beacon co-aligns all
4 (≈20 ms), but unicast traffic makes newest-frame spread bimodal (14–28 ms vs
250–711 ms), so 400 ms guard still fails.

(3) NDP self-trigger crashes the S3
Synchronized NDP self-trigger (null-data frame every 50 ms via esp_wifi_80211_tx(
WIFI_IF_STA) while promiscuous) crashes node1 and breaks node2's ESP-NOW sync (valid
flips false) — the wDev_ProcessFiq / SPI-flash-cache class of #396. Active TX while
promiscuous appears unsafe on S3; reverted.

QUESTIONS
(a) Is the ~0.28–1.5 s mesh-time residual a known bias/limitation of the ESP-NOW offset
estimate in c6_sync_espnow? Stable bias correction — or a reason node1 converges to
1.5 s off the leader?
(b) Intended way to get tight co-aligned captures on passive ESP32-S3 at ~60 ms —
beacon-anchored capture only (accepting ~10 Hz), or a sync mechanism I'm missing?
(c) Is esp_wifi_80211_tx sanctioned while promiscuous on S3? Mine crashes a node. If
not, what's the correct active-probe / NDP path on S3?

Happy to share my branch (Plan A + host robustness) and full capture logs. Thanks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the firmware-side c6_sync_espnow offset baseline and the mesh timestamp path, then compare its output with the Rust fusion timestamp and sync-valid admission logic. Reproduce using direct capture and scripts/udp-relay.py as described, while checking the reported 244 bin and 529 lib tests. Done requires identifying the cause of the residual timing error and S3 crash, or documenting a supported capture and synchronization path that meets the 60 ms guard.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
embedded-iot, networking
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.