anomalyco / anomalyco/opencode

[FEATURE]: framework signals for run-duration estimation (pre-flight estSec hits a ceiling)

Open
#46,755 0 comments 0 reactions 1 assignee View on GitHub

@neriousy is already working on this.

Since Sep 2, 2026.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Feature hasn't been suggested before.
  • I have verified this feature I'm about to request hasn't been suggested before.
Describe the enhancement you want to request
Context

External consumers of opencode care about one number: how long will this run take? I maintain one such consumer — an open-source desk device (Rubato_Plugins/opencode) that decides whether to interrupt the user: a 20s answer should never trigger a break; a 5-minute agent run should. So I built a zero-cost estimator: at chat.params, kNN over pre-flight features (model, context size, message length, tool mix) against locally stored samples with real outcomes, backfilled with actual token usage at step-finish for calibration.

It works — sort of. It is the weakest part of my integration, and I don't think I can fix it alone. Opening this to ask whether the framework can.

Where pre-flight estimation breaks down
  • Throughput variance: same feature vector, wildly different wall time depending on provider load, model version, streaming rate.
  • Tools dominate wall clock: in agentic runs most time is tool execution (builds, tests, network). Pre-flight features cannot see bash("npm test") coming; per-tool durations are their own distribution.
  • Retries, queues, rate limits: invisible at chat.params time, huge at runtime.
  • Cold start: per-machine sample stores are tiny — estimation is worst exactly where a new user starts.
Signals only the framework has
  1. Real prompt token count from the provider (usage in the first stream chunk / message_start) — far better than character-based estimates, available within the first seconds.
  2. TTFT and streaming rate — observable early, strongly correlated with remaining generation time.
  3. Per-tool duration history — opencode already sees every tool call and its wall time; a persisted per-tool (+ input-class) distribution (p50/p90) would make tool time estimable while the run is happening.
  4. Step/turn structure — planned tool calls before execution, enabling "steps remaining" progress instead of a single-point ETA.
Question for maintainers

Is any of this in scope for opencode itself? Three possible levels:

  1. Expose the raw signals (per-event timing, real token counts, per-tool wall time) — cheapest, high value; consumers do their own modeling.
  2. Aggregate: persist per-tool / per-model duration summaries locally and expose them to plugins.
  3. Native ETA: first-class estimated-remaining with confidence. I suspect full ETA is out of scope / unreliable, but levels 1–2 would let consumer-land finally build decent estimates.

My kNN + calibration dataset can serve as a consumer-side reference implementation — happy to share the feature set if useful.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.