cloudflare / cloudflare/agents
Investigate speculative programmatic tool calling in Code Mode
- Dominant language
- TypeScript
- Stars
- 5.6k
- Forks
- 711
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 53
Description
## Context
Alex Zhang describes speculative programmatic tool calling (sPTC), where a harness inspects a partially streamed program and starts safe tool calls before the model finishes generating the code. If the final program makes the call, execution reuses the in-flight or completed result. The same machinery can also run independent blocking calls concurrently.
This looks relevant to `@cloudflare/codemode`, where tool calls made by generated JavaScript can dominate the total runtime.
Sources:
- [Announcement on X](https://x.com/a1zhang/status/2091938825580716079)
- [Blog post](https://alexzhang13.github.io/blog/2026/spec-ptc/)
- [Reference implementation](https://github.com/alexzhang13/spec-ptc)
## What to investigate
- Where speculation would fit in Code Mode, and whether existing model/tool-call streaming APIs expose partial `code` arguments early enough.
- A safe eligibility contract for speculative calls, including purity, side effects, approvals, stable call identity, cancellation, and unused results.
- Whether an initial implementation should handle only calls with literal arguments rather than shadow-executing partial programs.
- How speculation would interact with the Worker sandbox, connectors, durable execution, and provider concurrency limits.
- The latency improvement, extra request cost, and correctness on representative Code Mode workloads.
## Expected outcome
Produce a small prototype or design note with measurements and a recommendation. If the approach is worthwhile, split the implementation and API work into follow-up issues.
Contributor guide
Assessment
This issue has not been assessed yet.