cloudflare / cloudflare/cloudflare-os

Long-lived open() RPC capabilities keep workspace Durable Objects billable

Open
#338 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
9.9k
Forks
1.2k
Avg merge
1d 20h
Merged PRs (30d)
107

Description

## Problem

A browser keeping a workspace open also keeps the workspace's `OverseerDurableObject.open()` RPC session open. Because `open()` returns an `Overseer` capability, the session remains in flight for the lifetime of that returned stub. Cloudflare bills the object for wall-clock duration while an RPC remains active, so an otherwise idle workspace can accrue continuous Durable Object duration until the tab navigates away or disconnects.

The associated `clientDisconnected` / `canceled` outcomes are normal capability-session teardown, but they appear in the Durable Object error bucket and obscure genuine failures.

## Production evidence

From one deployment's current billing period:

- total Durable Object duration: 534k GB-s
- `OverseerDurableObject`: 517k GB-s (96.8%)
- `AdminSettings`: 17.4 GB-s
- one recently used Overseer instance in the last 24h: 11k GB-s, 1k RPC requests, 28 errors (25 `clientDisconnected`), P50 request wall time 137s, zero alarm invocations

A controlled live-tail test opened the affected workspace for about four seconds and then navigated home. The resulting event was:

```json
{"entrypoint":"OverseerDurableObject","rpcMethod":"open","wallTime":4106,"outcome":"canceled"}
```

The invocation lifetime matched the browser's workspace-session lifetime. The workspace had no active agent at the time.

Cloudflare's pricing documentation says an RPC method returning an `RpcTarget` keeps subsequent capability calls in that same RPC session, and its limits documentation says a Durable Object remains active while an RPC is in flight.

## Current path

`AuthenticatedApiImpl.#openGadgetInternal()` calls `overseer.open(...)` and returns the resulting `Overseer` stub to the browser. The frontend correctly disposes the stub on navigation, but keeping a workspace tab open intentionally keeps it alive.

## Possible directions

1. Move the browser-to-workspace realtime channel to a Durable Object-hosted hibernatable WebSocket (`ctx.acceptWebSocket`) so an idle connected workspace can hibernate.
2. Replace the long-lived returned capability with short-lived RPC calls keyed by workspace/session identifiers, using subscriptions only while data is actually changing.
3. As a smaller mitigation, dispose/suspend the workspace capability after an inactivity or hidden-tab threshold and reconnect on activity. This reduces duration but does not remove the architectural cost while a visible idle tab remains open.
4. Separate normal capability disposal from actionable errors in observability so `clientDisconnected` does not dominate the incident signal.

A separate agent-turn watchdog gap was found during this investigation and is addressed in #337; it does not solve the ordinary `open()` session duration described here.

Contributor guide

Open the contributing guide

Research direction

Start at AuthenticatedApiImpl.#openGadgetInternal() and follow the call to OverseerDurableObject.open(...), then review the listed realtime-channel alternatives and the #337 boundary. Confirm the chosen approach against the observed billing and cancellation behavior; done means idle workspace sessions no longer remain continuously billable and normal capability teardown is separated from actionable errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend-api-design, cloud, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.