Parallel tool child cancellation leaves sibling tasks running past invocation completion

Aperta
#7,172 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

@llalitkumarrr ci sta già lavorando.

Dal 18/9/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

Problem

When one tool in a parallel tool batch undergoes child-originated cancellation, the batch can return without cancelling and awaiting its sibling tool calls. Here, child-originated cancellation means either the tool raises asyncio.CancelledError, or it cancels its own task with asyncio.current_task().cancel() and reaches an await. The caller does not cancel the batch.

With the public Runner, the invocation iterator finishes while a sibling tool is still executing. That sibling can subsequently perform a side effect. Callback ordering is an observable symptom; the core issue is that the parallel batch returns while a child task it spawned remains unsettled.

Minimal reproduction

  1. Use a development checkout of google/adk-python at f33d4923388a963d0c5cbf8f7855a88d255b0dea (also the latest main when checked on 2026-09-18), with its Python dependencies installed.
  2. Save the following as /tmp/adk-child-cancel.py.
  3. From the repository root, run PYTHONPATH=src:. python /tmp/adk-child-cancel.py using that environment.

This uses the repository's MockModel to supply one parallel function-call response. It does not use LiteLLM, a model API, or a network service.

import asyncio

from google.adk.agents import LlmAgent
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types
from tests.unittests.testing_utils import MockModel


async def main():
    trace = []
    started, release = asyncio.Event(), asyncio.Event()
    sibling_task = None

    async def stopper() -> dict:
        """Cancel this tool after its sibling starts."""
        await started.wait()
        trace.append("stopper:self_cancel")
        asyncio.current_task().cancel()
        await asyncio.sleep(0)  # Deliver this task's own cancellation.
        return {}

    async def sibling() -> dict:
        """Represent a delayed side effect."""
        nonlocal sibling_task
        sibling_task = asyncio.current_task()
        trace.append("sibling:start")
        started.set()
        try:
            await release.wait()
            trace.append("sibling:side_effect")  # Local append only.
            return {"ok": True}
        finally:
            trace.append("sibling:finally")

    async def after_agent(callback_context):
        trace.append("after_agent")

    async def after_tool(tool, args, tool_context, tool_response):
        trace.append("after_tool:" + tool.name)

    model = MockModel.create(responses=[[
        types.Part(function_call=types.FunctionCall(name=n, id=n, args={}))
        for n in ("stopper", "sibling")
    ]])
    agent = LlmAgent(
        name="probe", model=model, tools=[stopper, sibling],
        after_agent_callback=after_agent, after_tool_callback=after_tool,
    )
    service = InMemorySessionService()
    runner = Runner(app_name="scout", agent=agent, session_service=service)
    session = await service.create_session(app_name="scout", user_id="u")
    try:
        async for _ in runner.run_async(
            user_id="u", session_id=session.id,
            new_message=types.Content(
                role="user", parts=[types.Part(text="Run both tools.")]
            ),
        ):
            pass
        trace.append("runner_returned:complete")
        pending = sibling_task is not None and not sibling_task.done()
    finally:
        # Release/drain any surviving sibling so the repro itself leaks no task.
        release.set()
        if sibling_task is not None:
            await asyncio.gather(sibling_task, return_exceptions=True)
        await runner.close()

    print("\n".join(trace))
    print(f"sibling_pending_at_runner_return={pending}")


asyncio.run(main())
Observed output
sibling:start
stopper:self_cancel
after_agent
runner_returned:complete
sibling:side_effect
sibling:finally
after_tool:sibling
sibling_pending_at_runner_return=True

The sibling is gated with asyncio.Event, so this does not depend on timing sleeps. The only sleep(0) delivers the tool task's own cancellation. The local sibling:side_effect trace append stands in for an external write; the reproducer performs no external side effect and drains the surviving task before exiting.

Expected behavior

A parallel tool batch should not return while sibling tasks spawned by that batch are still running. If one child terminates the batch through cancellation, unfinished siblings should be cancelled and awaited before the original cancellation is propagated out of the batch.

The invariant is that sibling cleanup completes before the batch exits. This does not require converting cancellation into a normal tool error, changing callbacks to run from finally blocks, or changing how Runner exposes root-node cancellation.

Reproduction matrix

Condition Async: sibling pending at batch return Live: sibling pending at batch return
Normal completion no no
Tool raises ValueError no no
Caller cancels the entire batch no no
Tool raises CancelledError yes yes
Tool cancels its own task yes yes

Separate full Runner reproductions cover run_async with StreamingMode.NONE, run_async with SSE, and run_live with BIDI. All three exhibit the same child-cancellation behavior. Live uses the repository's MockModel, not a live model service.

Root cause and candidate repair

_gather_or_cancel catches Exception, cancels unfinished child tasks, awaits their cleanup, and re-raises. asyncio.CancelledError derives from BaseException, so child-originated cancellation bypasses that path. asyncio.gather() propagating a child's cancellation does not itself cancel its siblings.

A minimal candidate repair is to include asyncio.CancelledError in the existing cleanup path, for example with except (Exception, asyncio.CancelledError), then reuse the existing cancel/await/re-raise logic. I tested this as a process-local monkeypatch only; production source was not modified. Cancellation remains cancellation, rather than becoming a tool error response.

Verification and environment

Baseline: f33d4923 (2026-09-17), macOS arm64, Python 3.11.9.

  • Existing BaseAgent, async tool callback, and live tool callback suites: 111 passed.
  • Local scout matrix on unmodified main: 21 passed, 10 failed; all failures concern surviving siblings after child-originated cancellation.
  • Same matrix with the process-local candidate repair: 31 passed.
  • Stability: 40/40 deterministic reproductions (10 rounds × 2 tool execution modes × 2 child-cancellation forms).
  • Dependencies: google-genai 2.22.0, pydantic 2.12.5, pytest 9.1.1, pytest-asyncio 1.4.0.

The existing local environment was reused. Full supported-Python-version tests and real-backend live E2E have not been run. I have not established which release first introduced this behavior. I can provide the broader Runner probe and regression matrix if useful, and am happy to send a focused PR if this direction fits.

Duplicate check

I searched current issue/PR listings and cancellation-related terms and did not find the same case. Nearby reports differ:

  • #7058 / #7059: concurrent streaming tools with the same name overwrite task tracking. This repro uses two distinct ordinary async tools.
  • #6559 / #6560: OpenTelemetry context errors during early iterator close. This repro fully consumes the Runner iterator.
  • #4796: public stop-generating support. This report concerns cleanup of sibling tasks already spawned by a parallel tool batch.
Lingua principale
Python
Stelle
21.6k
Fork
4k
Merge medio
13h 49m
PR unite (30g)
10

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di google/adk-python

Tutte le issue di google/adk-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.