HarperFast / HarperFast/harper
Live-subscription re-auth sweep and admitted-target retention scale per subscription
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Problem
Two per-subscription costs landed with continuous re-authorization (#1414) and the operation-scoped read authorization restore (#1915). Under an MQTT fan-out workload (~150k+ subscriptions per worker; production Fabric nodes measured at 157,894 subs on one worker) they are significant:
1. **The 30s re-auth sweep scales per subscription.** `server/liveSubscriptionAuth.ts` runs `findAndValidateUser` + a deep `cloneRequestTarget` + `allowRead` serially for *every* registered subscription, every 30 seconds — even though a fan-out workload collapses to a handful of distinct (resource, user, target) admissions (29 distinct topics in the measured production case). At 150k+ subs/worker that is an estimated 0.5–1.5 cores of steady-state CPU across a 16-worker node, plus allocation churn feeding GC (measured at 26.4% of worker CPU in a production profile).
2. **`admittedTarget` is retained per subscription.** Since #1915 (cc8ebfb9e), every subscribe deep-clones the RequestTarget and the recheck closure retains it for the life of the subscription. Measured in a local load repro (1,000 WS-MQTT conns × 15 subs): RequestTarget count doubled 15,000 → 30,000, ≈ +0.4 KB/sub (~6 KB/conn, ~7% of subscription heap).
## Proposed fix (implemented and verified; full patch attached in the first comment)
Group registrations by a canonical admission identity — resource class (WeakMap'd id) + username + canonical target form. The sweep then runs one `findAndValidateUser` per username and one `allowRead` recheck per distinct admission; per-subscription work drops to a sync token-expiry comparison. Duplicate admitted-target clones are discarded at registration, so at most one snapshot is retained per distinct admission. Targets carrying reference-identity values (`rowFilter` functions, class instances, circular refs) have no safe canonical form and register ungrouped — preserving today's per-subscription behavior for them. Per-entry contexts still get `context.user` advanced to the fresh user before the group recheck runs.
The patch also includes two small delivery-loop items in `server/DurableSubscriptionsSession.ts`: yield every 100 delivered messages instead of allocating a promise + macrotask hop per message (3.5× loop overhead in fan-out benchmarks; real backpressure is the awaited listener/socket-drain, the yield is only event-loop fairness), and a shared frozen empty `checkPermission` object.
## Verification (local load repro: 1,000 WS-MQTT connections × 15 subscriptions, 29 topics, QoS 1)
- RequestTarget 30,000 → 15,029 (the 29 = one snapshot per distinct admission)
- 5.01 → 4.79 KB/sub worker heap
- sweep at a 3s test interval: ~0.1% of worker CPU (20 / 17,771 profile samples); per-sweep work goes from 15,000 × (validate + clone + allowRead) to 1 validate + 29 rechecks
- 24 new unit tests (grouping, fail-closed on validation/recheck errors, per-entry expiry within a group, teardown/unregister, key canonicalization + separator-collision resistance) and the full `integrationTests/security/subscription-revocation.test.ts` suite (6/6: drop_user, alter_role, token expiry × WS/MQTT) pass; oxlint + prettier clean.
Note for whoever picks this up: a cross-model review (per engineering guidelines) has not been run on the patch yet.
---
_Filed by an AI agent (Claude Code) from a subscription-path memory/CPU investigation; numbers from a production 5.1.26 Fabric heap snapshot + CPU profile and a local 15k-subscription load repro against this repo._
Contributor guide
Research direction
Start by reviewing server/liveSubscriptionAuth.ts for the subscription re-auth sweep and server/DurableSubscriptionsSession.ts for the delivery-loop changes. Compare the attached implementation with the stated grouping, retention, expiry, teardown, and fail-closed requirements, then run integrationTests/security/subscription-revocation.test.ts and the repository's oxlint and prettier checks; done means the reported load and test results are preserved after cross-model review.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nodejs, typescript
- Domain
- backend, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100