electric-sql / electric-sql/electric

agents-server: webhook dispatch dropped on service-routed durable-streams deployments

Open
#4,357 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
10.4k
Forks
375
Avg merge
3d 1h
Merged PRs (30d)
18

Description

## Symptom

On service-prefixed / service-routed durable-streams deployments (e.g. stratovolt's `cloud-agents-server`, verified against `https://agents.electric-sql.cloud/?service=svc-...`):

1. Register an entity type with `default_dispatch_policy.targets = [{ type: 'webhook', url: 'https:///wake' }]`.
2. `PUT /_electric/entities//` → 201 (entity stored with the webhook dispatch policy).
3. `POST /_electric/entities///send` → 204 (inbox event appended).
4. **No HTTP POST ever lands on the webhook URL.**

Pull-wake dispatch against the same tenant works end-to-end, so this is webhook-specific. The public webhook URL is reachable; the cloud just never calls it.

## Root cause

Two-tier path-namespace mismatch on the subscription path that doesn't exist on the data path:

- **Stream appends** are routed through the `DurableStreamsRoutingAdapter` (e.g. `serviceRoutedDurableStreamsAdapter` on the cloud), so they land in the worker keyed on the service-prefixed backend path (`///main`).
- **Subscription registrations** went through `StreamClient.backendSubscriptionPath` (`packages/agents-server/src/stream-client.ts:222–224`), which only slash-normalised. So the subscription's `streams` (and `pattern`, `wake_stream`, ack/release `stream`/`path` fields) were stored as the un-prefixed path (`//main`).

The worker's HotBuffer lookup keyed on the append's resource id never matched the subscription link, and webhook fanout silently dropped.

Pull-wake worked only because the runner writes a wake notification directly to its own `wake_stream` and reads it back — no HotBuffer fanout coordination needed, so the path mismatch never mattered.

## Suggested fix

`StreamClient` should accept the `DurableStreamsRoutingAdapter` (and a `serviceId`) and use `toBackendStreamPath` / `toRuntimeStreamPath` on subscription payloads — symmetrically with how the per-request `durable-streams-router.ts` rewrites stream URLs. `AgentsHostTenantConfig` should gain a matching option so cloud callers can pass the adapter at tenant-registration time.

PR: #4356

## Reproduction

End-to-end repro lives in https://github.com/electric-sql/OpenFactory `packages/discord-bot` — registering a `discord-bot` entity type with a webhook dispatch target reproduces 100% on the hosted cloud tenant. Switching the same code to pull-wake (`{ type: 'runner', runnerId }`) works.

Originally filed at electric-sql/stratovolt#1519.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.