github / github/copilot-cli

Parallel tool calling non-deterministic response order results in confused bots

Aperta
#4,420 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area:tools
Lingua principale
Shell
Stelle
11.2k
Fork
1.9k
Merge medio
14h 16m
PR unite (30g)
6

Descrizione

### Describe the bug

Copilot harness loses request correlation for parallel tool calls

## Summary

The Copilot harness does not preserve a reliable association between parallel tool requests and their responses.

The harness can return parallel responses without the original request, a caller-defined alias, or another usable
correlation value. The agent can then associate a response with the wrong request.

## Observed behaviour

The agent sent separate service searches for these filters:

```json
{ "type": ["application"] }
{ "type": ["data-product"] }
{ "type": ["kafka-connect"] }
```

The responses contained different totals. The harness did not return a request alias or the original request with each response. The agent matched the totals to the wrong filters and reported an incorrect breakdown.

The response position was not a safe correlation method.

Image

## Expected behaviour

The harness must preserve the association for every parallel invocation.

The parallel call format can accept an optional caller-defined alias:

```json
{
"request_alias": "kafka-connect-count",
"tool": "delivery_platform_search_services",
"arguments": {
"type": ["kafka-connect"],
"environment": "prod"
}
}
```

Each response must return the alias and the request:

```json
{
"request_alias": "kafka-connect-count",
"request": {
"type": ["kafka-connect"],
"environment": "prod"
},
"result": {
"pagination": {
"total_items": 71
}
}
}
```

The harness must not require response order to match request order.

If the caller does not provide an alias, the harness can return its internal call ID. The internal ID alone is not
enough for semantic correlation. The response must also include the original request or another stable request
description.

## Suggested fix

Add request metadata to the parallel tool runner.

The runner must:

1. Accept an optional `request_alias` for each invocation.
2. Keep the alias with the invocation while the tool runs.
3. Return the alias, original arguments, and result together.
4. Preserve the association when responses complete out of order.
5. Keep aliases out of the arguments sent to the underlying tool API.

The same response envelope can apply to single tool calls for consistent behaviour.

## Workaround

I did a poor-mans implementation of the above, by wrapping our `server.registerTool` function with something that:

- Adds an optional `request_alias` field to registered tool input schemas.
- Removes `request_alias` before the tool handler runs.
- Adds the alias to `structuredContent.request_alias` in the response.
- Extends declared output schemas when they exist.
- Covers JSON, text, generated, and proxied tools.

I also added the following to our instruction file:

- Assign a unique alias to every parallel invocation.
- Put the alias in the actual tool arguments.
- Match responses by alias and exact arguments.
- Run calls sequentially when a tool does not support aliases.
- Treat a response without the expected alias as uncorrelated.

That worked well:

Image

But only works for tools we register.

### Affected version

`1.0.78`

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia tracciando il parallel tool runner e il wrapper server.registerTool descritti nella soluzione alternativa. Verifica come i metadati dell’invocazione vengono trasferiti nelle risposte e come gli argomenti vengono passati all’API dello strumento sottostante. Il lavoro è completato quando alias, richieste originali e risultati restano associati anche se le chiamate vengono completate fuori ordine, senza inoltrare gli alias come argomenti dello strumento.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
shell
Ambito
cli, tooling
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
55/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.