cloudflare / cloudflare/cloudflare-os
Long-lived open() RPC capabilities keep workspace Durable Objects billable
- Dominant language
- TypeScript
- Stars
- 9.9k
- Forks
- 1.2k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 107
Description
## Problem
A browser keeping a workspace open also keeps the workspace's `OverseerDurableObject.open()` RPC session open. Because `open()` returns an `Overseer` capability, the session remains in flight for the lifetime of that returned stub. Cloudflare bills the object for wall-clock duration while an RPC remains active, so an otherwise idle workspace can accrue continuous Durable Object duration until the tab navigates away or disconnects.
The associated `clientDisconnected` / `canceled` outcomes are normal capability-session teardown, but they appear in the Durable Object error bucket and obscure genuine failures.
## Production evidence
From one deployment's current billing period:
- total Durable Object duration: 534k GB-s
- `OverseerDurableObject`: 517k GB-s (96.8%)
- `AdminSettings`: 17.4 GB-s
- one recently used Overseer instance in the last 24h: 11k GB-s, 1k RPC requests, 28 errors (25 `clientDisconnected`), P50 request wall time 137s, zero alarm invocations
A controlled live-tail test opened the affected workspace for about four seconds and then navigated home. The resulting event was:
```json
{"entrypoint":"OverseerDurableObject","rpcMethod":"open","wallTime":4106,"outcome":"canceled"}
```
The invocation lifetime matched the browser's workspace-session lifetime. The workspace had no active agent at the time.
Cloudflare's pricing documentation says an RPC method returning an `RpcTarget` keeps subsequent capability calls in that same RPC session, and its limits documentation says a Durable Object remains active while an RPC is in flight.
## Current path
`AuthenticatedApiImpl.#openGadgetInternal()` calls `overseer.open(...)` and returns the resulting `Overseer` stub to the browser. The frontend correctly disposes the stub on navigation, but keeping a workspace tab open intentionally keeps it alive.
## Possible directions
1. Move the browser-to-workspace realtime channel to a Durable Object-hosted hibernatable WebSocket (`ctx.acceptWebSocket`) so an idle connected workspace can hibernate.
2. Replace the long-lived returned capability with short-lived RPC calls keyed by workspace/session identifiers, using subscriptions only while data is actually changing.
3. As a smaller mitigation, dispose/suspend the workspace capability after an inactivity or hidden-tab threshold and reconnect on activity. This reduces duration but does not remove the architectural cost while a visible idle tab remains open.
4. Separate normal capability disposal from actionable errors in observability so `clientDisconnected` does not dominate the incident signal.
A separate agent-turn watchdog gap was found during this investigation and is addressed in #337; it does not solve the ordinary `open()` session duration described here.
Contributor guide
Research direction
Start at AuthenticatedApiImpl.#openGadgetInternal() and follow the call to OverseerDurableObject.open(...), then review the listed realtime-channel alternatives and the #337 boundary. Confirm the chosen approach against the observed billing and cancellation behavior; done means idle workspace sessions no longer remain continuously billable and normal capability teardown is separated from actionable errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend-api-design, cloud, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100