Data-quality report: v1beta3 crypto orderbook stream (`us-1`) serves stale ("zombie") levels in snapshots; per-pair delta pipelines drop removals

**Date observed:** 2026-08-09 (all timestamps UTC)
**Feed:** `wss://stream.data.alpaca.markets/v1beta3/crypto/{loc}` with `loc=us-1` (Kraken US), `orderbooks` channel, plain-JSON transport
**Also cross-checked against:** the same feed's `quotes` and `trades` channels, and `GET /v1beta3/crypto/us-1/bars`
**Affected pairs observed:** BTC/USD, ETH/USD, SOL/USD, XRP/USD, DOGE/USD, LINK/USD (LTC/USD, AVAX/USD, DOT/USD, SHIB/USD were clean during the same sessions)

## Summary

The `orderbooks` channel's `r: true` snapshot routinely contains **internally crossed books**: price levels that were plainly removed from the live book long ago (in some cases hours ago) are still present in every fresh snapshot, alongside current levels. The same stale levels reappear **byte-identically across repeated subscriptions spanning more than an hour**, and no update message ever removes them. Separately, for at least two pairs the **bid-side delta stream stopped delivering entirely** while the ask side continued normally. The `quotes` channel remained accurate throughout — the orderbook channel contradicts your own NBBO.

We maintain a client-side book exactly per your documentation (seed on `r: true`, upsert on updates, delete on `s: 0`) with an integrity check that refuses crossed state, which is how this surfaced immediately.

## Finding 1 — snapshots are internally crossed as sent

Raw 15-second capture starting 22:16:57Z (wscat-equivalent, JSON): the first message per pair after subscribe was `r: true` with 100 bids + 100 asks. For five of five subscribed pairs the snapshot was crossed **within itself**:

| Pair | Snapshot best bid | Ask levels at/below best bid | Lowest such ask | First ask above best bid |
| -------- | ----------------- | ---------------------------- | --------------- | ------------------------------------------------------------ |
| BTC/USD | 65338.3 | **55 of 100** | 64366.7 | 65338.4 (one-tick spread — the fresh top is self-consistent) |
| LINK/USD | 8.37096 | 34 of 100 | 8.2351 | 8.37102 |
| DOGE/USD | 0.0708519 | 14 of 100 | 0.0707299 | 0.0708651 |
| SOL/USD | 77.56 | 1 | 74.93 | 77.57 |
| XRP/USD | 1.04612 | 1 | 1.03055 | 1.04614 |

The concurrent `quotes` channel NBBO agreed with the snapshot's _fresh_ top in every case. BTC/USD never quoted below 65069.4 bid in our whole recorded session — an ask at 64366.7 was roughly 1.2% below the market and had not been a plausible price for hours. No snapshot entry carried `s: 0`; these arrive as live levels with sizes.

## Finding 2 — the stale levels are persistent server-side state, not lag

The identical prices recur in every fresh snapshot across at least five separate subscriptions over ~80 minutes (21:59Z, 22:15Z, 22:16Z, 22:57Z, 23:17Z): BTC/USD ask 64366.7, SOL/USD ask 74.93, XRP/USD ask 1.03055, LINK/USD ask 8.2351 — byte-identical each time. Across the 15-second capture, **zero** update messages touched any of these levels (no size change, no `s: 0` removal). They appear to be levels consumed/cancelled as the market moved through them whose removal messages were dropped, and the snapshot builder never prunes them.

They also accumulate mid-session: ETH/USD ran clean from 21:59Z until 22:14:21Z, when a bid update at 1926.95 crossed a resting ask at 1926.66 that (per your own quotes channel) no longer existed — and from then on 1926.66 appeared as a zombie in every subsequent ETH/USD snapshot.

Notably, the snapshot for BTC/USD served at ~23:03Z still carried a _bid_ ladder topping at 65252.2 while your quotes channel had the market at ~64,960 — i.e., by then the snapshot's bid side had gone stale as well. The snapshot content appears frozen at an earlier book state plus accumulated dirt, rather than a live view.

## Finding 3 — bid-side delta pipelines can die entirely per pair

At ~22:57Z, after a ~1% decline, the DOGE/USD and LINK/USD snapshots' **entire 100-level bid ladders** were above the concurrent NBBO ask (all stale). After discarding those, we received **zero bid-side update messages for over four minutes** on both pairs, while ask-side updates continued at normal rates and the quotes channel kept publishing fresh bids. The bid half of the orderbook pipeline for those pairs appears to have stopped publishing, which would also explain how snapshots accumulate stale bid ladders.

## Reproduction

```
wscat -c wss://stream.data.alpaca.markets/v1beta3/crypto/us-1
> {"action":"auth","key":"...","secret":"..."}
> {"action":"subscribe","orderbooks":["BTC/USD","SOL/USD","XRP/USD","DOGE/USD","LINK/USD"],"quotes":["BTC/USD"]}
```

Then, on the first `r: true` message per pair: compare `max(b[].p)` against `min(a[].p)` — we observed `min(a[].p) < max(b[].p)` (a crossed book) on every subscription for the affected pairs, and the specific stale prices listed above recur across sessions. Comparing against the concurrent `q` messages shows which side is stale. We can provide the full raw JSON capture (4,922 orderbook messages with timestamps) on request.

## Impact

Any client that builds a book from this channel per the documented semantics ends up with a crossed, partially frozen book: best-ask (or best-bid) is wrong by up to several percent, and depth/imbalance calculations are unusable on affected pairs. Clients without an integrity check will silently consume these values.

## Questions / requests

1. Can the snapshot builder prune levels whose removals were dropped (or rebuild from the venue's current book) so `r: true` reflects live state?
2. Is the per-pair, per-side pipeline stall (Finding 3) a known failure mode, and is there monitoring on your side for it?
3. Is there any way for clients to request a fresh snapshot mid-session other than unsubscribe/resubscribe — and any integrity aid (e.g., a checksum, as the upstream venue's native feed provides)?