cloudflare / cloudflare/agents

Investigate speculative programmatic tool calling in Code Mode

Open
#2,159 0 comments 0 reactions 1 assignee Claimed by @mattzcarey View on GitHub
enhancement
Dominant language
TypeScript
Stars
5.6k
Forks
711
Avg merge
1d 20h
Merged PRs (30d)
53

Description

## Context

Alex Zhang describes speculative programmatic tool calling (sPTC), where a harness inspects a partially streamed program and starts safe tool calls before the model finishes generating the code. If the final program makes the call, execution reuses the in-flight or completed result. The same machinery can also run independent blocking calls concurrently.

This looks relevant to `@cloudflare/codemode`, where tool calls made by generated JavaScript can dominate the total runtime.

Sources:

- [Announcement on X](https://x.com/a1zhang/status/2091938825580716079)
- [Blog post](https://alexzhang13.github.io/blog/2026/spec-ptc/)
- [Reference implementation](https://github.com/alexzhang13/spec-ptc)

## What to investigate

- Where speculation would fit in Code Mode, and whether existing model/tool-call streaming APIs expose partial `code` arguments early enough.
- A safe eligibility contract for speculative calls, including purity, side effects, approvals, stable call identity, cancellation, and unused results.
- Whether an initial implementation should handle only calls with literal arguments rather than shadow-executing partial programs.
- How speculation would interact with the Worker sandbox, connectors, durable execution, and provider concurrency limits.
- The latency improvement, extra request cost, and correctness on representative Code Mode workloads.

## Expected outcome

Produce a small prototype or design note with measurements and a recommendation. If the approach is worthwhile, split the implementation and API work into follow-up issues.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.