cloudflare / cloudflare/workers-sdk
🚀 Feature: support remote Hyperdrive bindings in local dev (raw TCP relay)
- Dominant language
- TypeScript
- Stars
- 4.5k
- Forks
- 1.5k
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 186
Description
## Which Cloudflare product(s) does this pertain to?
Wrangler, Miniflare (remote bindings), Hyperdrive
## Motivation
Hyperdrive bindings are currently `local-only` in local dev: Miniflare connects directly to the
origin database via `localConnectionString`. For databases that are only reachable through a
private network — e.g. Hyperdrive configs whose origin is a tcp-type VPC Service (#12874) — there
is nothing a developer can put in `localConnectionString`, so local dev is simply not possible.
Now that raw TCP `connect()` tunnelling over remote bindings has landed for VPC Networks/Services
(#14712), the same relay can carry Hyperdrive traffic. @xortive suggested exactly this in
https://github.com/cloudflare/workers-sdk/pull/14712#issuecomment-5050404309. We prototyped
it end-to-end against the real edge and are proposing a design here before sending a PR
(per CONTRIBUTING's discuss-first guidance), because one part of the semantics deserves
maintainer input.
## What we verified empirically (all against a real preview session + real private MySQL)
1. **Preview sessions already provision Hyperdrive raw bindings.** The remote-bindings pipeline is
type-agnostic; `startRemoteProxySession({ HYPERDRIVE: { type: "hyperdrive", id } })` works today.
2. **The edge side needs no changes.** `env.HYPERDRIVE.connect()` in the ProxyServerWorker context
returns a socket to Hyperdrive's MySQL front-end; the address argument is ignored (same
config-decides-routing model as VPC Services), so the existing `MF-Connect-Address` tunnel
endpoint from #14712 works unmodified.
3. **`connectionString` is fully synthetic on both ends**: magic `.hyperdrive.local` host,
**per-session generated dummy credentials**, and the config id in the database slot. The client
authenticates to Hyperdrive with these; Hyperdrive holds the real origin credentials.
4. **Full path proven**: local TCP bridge → WebSocket relay (the #14712 tunnel, unmodified) →
edge Hyperdrive → real origin. A stock `mysql2` pool authenticated successfully and
`SELECT VERSION()` returned the **origin server's** version (8.0.25) — i.e. post-auth queries
are proxied to the real database, exactly like production.
## Two constraints that shape the design
**(a) The Hyperdrive designator must stay `external` (tcp).** Our first prototype pointed the
Hyperdrive binding's designator at the shared remote-proxy-client Worker (the D1/VPC pattern) —
workerd **segfaults** in that configuration (crash in the Hyperdrive JSG `connect()` path when the
designator targets a Worker service; minimal standalone repro filed as cloudflare/workerd#6901). The safe local design therefore mirrors Miniflare's existing Hyperdrive SSL-proxy
pattern: keep the designator as `external.tcp → 127.0.0.1:` and run a small local TCP bridge
on that port which relays bytes over the remote-proxy WebSocket. Verified: no crash, clean relay.
**(b) Credentials must be seeded from the edge session.** The edge generates fresh dummy
credentials per session, so the local binding's `connectionString` must reproduce the **edge
session's** user/password/database (config id) or drivers fail auth after the greeting. Our
prototype adds a tiny guarded endpoint to ProxyServerWorker (`MF-HD-Seed` + `MF-Binding` headers →
returns the binding's `connectionString` once, no logging) and injects those values into the local
binding after the session is established. This is the only genuinely new seam: it needs one async
step in the dev pipeline after `maybeStartOrUpdateRemoteProxySession` resolves (the current
`buildMiniflareBindingOptions` path is synchronous).
## Proposed semantics (maintainer input wanted)
How should users opt in? Options, in our order of preference:
1. **`remote: true` on the hyperdrive binding**, consistent with other bindings. When set,
`localConnectionString` is not required (and ignored); when unset, behavior is exactly today's.
2. Same as (1) but keep `localConnectionString` as an explicit fallback if the remote session
cannot be established (more moving parts; unclear it's wanted).
3. Gate the whole thing behind an `--experimental-*` flag first.
Scope of the change (all TypeScript, no workerd changes): miniflare hyperdrive plugin
(schema + bridge service via the existing `HyperdriveProxyController` pattern), the ProxyServerWorker
seed endpoint, `hyperdriveEntry`/`pickRemoteBindings`/config validation in wrangler
(`hyperdrive: "local-only"` → conditional remote), plus the async credential-seeding step.
We're happy to send the PR once there's agreement on the semantics. Tests would extend the
existing `remote-bindings-connect.spec.ts` harness.
## Reproduction / evidence
Spike code is local-only, but the key measurements (masked) were:
```
SEED: db= userLen=32 passLen=32 ssl=disabled
AUTH: GREEN (connection established)
QUERY-RESULT: [{"one":1,"version":"8.0.25"}] # origin version, not the proxy banner
```
and the edge probe of `env.HYPERDRIVE`:
```
connectionString ≈ mysql://:@.hyperdrive.local:3306/?ssl-mode=disabled
connect()/connect("host:port") → identical MySQL handshake (address ignored)
```
Contributor guide
Research direction
Start with the existing remote-bindings-connect.spec.ts harness and the HyperdriveProxyController pattern in Miniflare. Trace hyperdriveEntry, pickRemoteBindings, config validation, buildMiniflareBindingOptions, and maybeStartOrUpdateRemoteProxySession in Wrangler to understand the synchronous-to-async boundary. Done means an agreed remote opt-in, seeded session credentials, a local TCP relay, and tests covering a successful remote Hyperdrive connection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, typescript
- Domain
- cli, databases, devtools
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100