.NET: [Feature]: Native task budget for the agent loop — advisory token countdown with graceful wrap-up

Description

Summary

Add a first-class task budget to the MAF agent loop: an advisory token countdown surfaced to the model on each model call and accounted across the whole loop (thinking + tool calls + tool results + output), plus an optional graceful wrap-up backstop when the budget is exhausted. Opt-in and non-breaking. A small sample implementation (linked below) exists only to make this request concrete — it is an illustrative proof of concept, not a library to adopt, depend on, or merge as-is.

Problem / motivation (the core reason)

Enterprises now apply FinOps to generative AI: spend must be governed and justified per agent run, and maximum agent effort is not always the business optimum. On open-ended tasks, extra loop iterations eventually stop adding value — and can make the result worse (over-processed output, the "lost in the middle" effect). A real customer example: a design-generation agent assisting human designers produced its best result within a bounded amount of work; letting the loop run long overshot the "sweet spot" into an over-engineered result — worse for the user and more expensive. Capping the loop with a token budget keeps runs in the sweet spot: predictable cost and better output.

max_iterations / turn caps are a poor proxy for this (one 128k-context turn ≠ ten 1k turns), and a hard max_tokens cutoff truncates mid-action and leaves partial state. What developers need is a token budget the model can see and pace itself against, then finish gracefully.

What the request looks like in practice (sample implementation)

To make the request concrete, I built a small sample — hatasaki/agent-framework-task-budget (against agent-framework python v1.10). It exists purely to illustrate the desired behavior and to surface the workarounds a native design would remove — it is not proposed as a package to depend on or to be merged. Measured on a hosted/remote MAF agent (Foundry-hosted gpt-5.4-mini) in advisory mode, budget set to ~40% of the task's natural cost, on a genuine multi-step tool loop:

	advisory OFF	advisory ON
tool iterations	12	6
total tokens	8,955	5,216
outcome	ran to completion	model chose to stop early + wrapped up

Validation note: the measured results above reflect advisory mode only — the savings come purely from the model self-pacing against the countdown. The optional enforce (graceful wrap-up backstop) is described in the Proposal as the intended on-exhaustion behavior, and is not part of these numbers. (Honest caveat: the self-pacing benefit shows up on genuine multi-step loops; on trivial 1–2 tool-call tasks the countdown text is just extra input.)

Prior art — an established governance pattern (incl. Microsoft's own)

Budget/turn/cost control of the agent loop is a well-established, cross-framework pattern, and MAF already has adjacent pieces:

• Microsoft Agent Governance Toolkit — TokenBudgetTracker (per-session token limits with warnings).
• MAF itself already accounts tokens/turns in the compaction framework (CompactionTriggers.TokensExceed / TurnsExceed).
• In-repo demand: #4142 ("Cost/Token Circuit Breakers") and PR #5073.
• Other agent SDKs expose loop caps too (e.g. OpenAI Agents SDK max_turns / MaxTurnsExceeded; Anthropic task budgets offer an advisory, model-visible, graceful-finish variant).

In short, this is not a novel or vendor-specific idea — it's a common governance primitive that MAF users need and that MAF currently lacks as a first-class concept.

Relationship to #4142 / #5073 (complementary — can subsume it)

#4142 / #5073 propose a ChatMiddleware circuit breaker that hard-halts on a cap (e.g. an exception). That's a valuable fail-safe. This request differs and can subsume it:

• Advisory & model-visible by default — the model self-paces, which is what actually reduces spend (fewer iterations), not just external enforcement.
• Graceful wrap-up, not an exception — the loop stops between turns and keeps the model's best partial answer.
• First-class, not an add-on — a run-level concept with consistent semantics across providers, across in-process and remote/hosted invocation, and both SDKs.

A native design can offer advisory self-pacing as the default, with an optional enforcing mode that covers the #4142 hard-stop behavior.

Why this belongs in the core (not a standalone extension)

Even this minimal sample can only deliver the behavior by (a) injecting the countdown via middleware and (b) shipping a hosting shim (budget_responses_host) because the stock Responses host drops the request metadata that carries the budget. Those workarounds exist only because there is no native concept — and they bite hardest on the remote/hosted path, which is the common production pattern. The goal is not to adopt the sample, but to make the capability native. A native task budget would:

• remove those workarounds (native accounting; native model-visible budget; hosting that carries/honors the budget without a metadata shim);
• provide one canonical, provider-agnostic mechanism instead of every team re-implementing it;
• be usable both in-process (a run option — the fundamental MAF usage, which the sample does not yet cover) and remotely (a first-class budget field for hosted agents); and
• compose with usage/telemetry (OpenTelemetry), hosting, and later workflows / multi-agent runs.

Proposal

• A token budget accumulated across the whole loop (thinking + tool calls + tool results + output), extensible to estimated cost.
• Advisory by default — surface remaining budget to the model each model call so it self-regulates.
• Optional enforce mode — a graceful wrap-up backstop that short-circuits the next tool call and asks the model to finalize from what it has (never truncates to empty).
• A stop_reason distinguishing normal completion from a budget-driven wrap-up, surfaced through usage/telemetry.
• Expressible remotely (behavior illustrated by the sample — a first-class budget field for hosted agents) and in-process (a run option), with parity across Python and .NET.

Validation scope (transparency)

The behavior and savings above were measured on the sample's remote/hosted path in advisory mode. The in-process run-option shown in the Code Sample is a proposed native surface, not present in the sample; native integration should make it straightforward (in-process, MAF owns the loop directly, so no metadata plumbing is needed).

Open design questions

• Remote transport: should a remote caller pass the budget via a native hosting field the host preserves and honors, instead of relying on Responses metadata (which the stock host currently drops)?
• In-process surface: run-option shape for the in-process path.
• Naming/shape (task_budget / TaskBudget, enforce, stop_reason values), and whether a budget can be shared across a workflow / multi-agent run.

Compatibility & scope

• Opt-in and non-breaking (default unlimited → current behavior preserved).
• Start with single-agent runs (Python + .NET parity), then workflows.
• The linked repo is a sample for reference only. The API in the Code Sample is a strawman.

Code Sample

# Strawman only — names/shape TBD.
#
# ── A) Remote / hosted — the common production path ───────────────────────────
#    (behavior illustrated by the sample implementation; advisory mode only)
# With a native design, the server collapses to standard hosting
# (no enable_task_budget / budget_responses_host wiring) and the budget is
# honored natively. The remote caller sets the budget per request:

from openai import OpenAI
client = OpenAI(base_url="https://<your-hosted-agent-endpoint>", api_key="...")

client.responses.create(
    model="my-agent",
    input="Investigate the flaky CI test and propose a fix.",
    metadata={
        "task_budget_tokens": "80000",   # advisory countdown across the loop
        "task_budget_enforce": "true",   # optional: graceful wrap-up backstop
    },
    # stream=True is honored too
)
# Open question: expose this remote budget as a first-class hosting field
# rather than via metadata.



# ── How the SAMPLE illustrates it today (reference only — not a package to adopt) ──
#    (verified, remote; measured savings are advisory-only, see Description)

from agent_framework_task_budget import enable_task_budget, budget_responses_host

enable_task_budget(agent)            # advisory countdown + token accounting (middleware)
app = budget_responses_host(agent)   # hosting shim: keeps request metadata from being dropped

# Client side (OpenAI SDK Responses API; imports nothing from the sample):
client.responses.create(
    model="my-agent",
    input="Investigate the flaky CI test.",
    metadata={"task_budget_tokens": "80000", "task_budget_enforce": "true"},
)



# ── B) In-process — proposed native surface ───────────────────────────────────
#    (NOT in the sample; requested for parity — the fundamental MAF usage)

result = await agent.run(
    "Investigate the flaky CI test and propose a fix.",
    task_budget=TaskBudget(
        tokens=80_000,   # advisory countdown surfaced to the model each model call
        enforce=False,   # False (default) = advisory self-pacing;
                         # True = graceful wrap-up backstop (finalize from what it has)
    ),
)
print(result.usage.total_tokens, result.stop_reason)   # "completed" | "budget_wrapup"



// ── B) In-process (.NET) — same concept via AgentRunOptions ───────────────────
//    (proposed native surface; for parity)

var response = await agent.RunAsync(
    "Investigate the flaky CI test and propose a fix.",
    options: new AgentRunOptions
    {
        TaskBudget = new TaskBudget { Tokens = 80_000, Enforce = false }
    });
Console.WriteLine($"{response.Usage.TotalTokens} tokens; stop: {response.StopReason}");

Language/SDK

Both