cloudwego / cloudwego/eino

panic: send on closed channel in typedAgentTool.InvokableRun (adk/agent_tool.go:243) after cancellation — path missed by #929

Open Beginner friendly
#1,236 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
13k
Forks
1.1k
Avg merge
4h 6m
Merged PRs (30d)
41

Description

## Summary

`panic: send on closed channel` crashes the whole process when a context is
cancelled while an in-flight agent-as-tool call (`typedAgentTool`, e.g. the
`task` tool of a deep agent) is still running. When the sub-agent finishes
after the parent layer has already closed its event generator, the event
forwarding loop in `typedAgentTool.InvokableRun` calls `gen.Send(event)` on a
closed `AsyncGenerator` and panics.

PR #929 fixed the same panic shape for the `chatModelAgentExecCtx.send` path
(`isImmediateCancelled()` fast path + `trySend()` safety net), but the
`agent_tool.go` event forwarding path was not covered by that fix.

## Environment

- eino v0.9.13 (goroutine stack below)
- Also checked v0.9.19 (latest stable) and current `main`: `adk/agent_tool.go:243`
still calls `gen.Send(event)` and `internal/channel.go:42` still panics on
closed — unchanged.

## Stack trace

```
panic: send on closed channel

goroutine 810 [running]:
github.com/cloudwego/eino/internal.(*UnboundedChan[...]).Send(...)
.../eino@v0.9.13/internal/channel.go:42
github.com/cloudwego/eino/adk.(*AsyncGenerator[...]).Send(...)
.../eino@v0.9.13/adk/utils.go:44
github.com/cloudwego/eino/adk.(*typedAgentTool[...]).InvokableRun(...)
.../eino@v0.9.13/adk/agent_tool.go:243 +0x9c8
github.com/cloudwego/eino/adk/prebuilt/deep.(*typedTaskTool[...]).InvokableRun(...)
.../eino@v0.9.13/adk/prebuilt/deep/task_tool.go:174 +0x1dc
github.com/cloudwego/eino/compose.(*invokableToolWithCallback).InvokableRun(...)
.../eino@v0.9.13/compose/tool_node.go:661 +0x8c
... (compose tool_node / adk wrappers middleware chain) ...
```

## Timeline

Observed in a production deep-agent orchestration (3 layers: supervisor →
sector commander → workers, all connected via agent-as-tool). Cancelling the
run while task delegations are in flight:

```
Goroutine A (run owner) Goroutine B (in-flight tool call)
──────────────────────────── ─────────────────────────────────
ctx cancelled (user stop)
graph converges, logs:
[GraphRunError] context has been canceled
parent layer generator closed
sub-agent iteration returns events
typedAgentTool.InvokableRun loop:
gen.Send(event) → PANIC
```

The log line immediately before the panic is the graph-level cancellation
error, i.e. the run had already converged on the consumer side — the producer
side (agent_tool event forwarding) then hit the closed channel ~milliseconds
later. The wider the fan-out of concurrent agent-as-tool calls, the easier
this race is to hit on stop/cancel (in our case: several in-flight task
delegations at once).

## Root cause

1. `internal/channel.go:42` — `UnboundedChan.Send` panics when closed
(`TrySend` in the same file already returns `false` instead).
2. `adk/agent_tool.go:243` — the event forwarding loop of
`typedAgentTool.InvokableRun` uses the panicking `gen.Send(event)`.

Notably, `AsyncGenerator.trySend()` already exists
(`adk/utils.go:47`, added by #929) and is exactly the safe variant needed
here — it is just not used on this path.

## Suggested fix

In `adk/agent_tool.go:243`, replace the bare send with the safe variant and
drop the event when the generator is already closed (the consumer is gone by
definition in this race):

```go
if gen != nil {
if event.Action == nil || event.Action.Interrupted == nil {
// ...
if !gen.trySend(event) {
break // generator closed after cancellation: no consumer left
}
}
}
```

(or apply the same two-layer guard style as #929 if a fast-path cancellation
check is preferred.)

## Workaround

For anyone hitting this before a fix lands: wrapping tool calls with a
`compose.ToolMiddleware` that `defer recover()`s and converts the panic into
a `ctx.Err()`-wrapped tool error keeps the process alive, since the middleware
sits on the call stack above `typedAgentTool.InvokableRun`.

Contributor guide

Open the contributing guide

Research direction

Start in adk/agent_tool.go around line 243 and compare its event-forwarding loop with the safe sending path in adk/utils.go and the cancellation fix from #929. Reproduce or add coverage for cancellation during an in-flight agent-as-tool call, then verify the closed generator causes no panic and the run completes with its cancellation error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ai-infra-agents
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.