agentscope-ai / agentscope-ai/agentscope-java

MemoryFlushMiddleware delays agent event stream completion, causing perceived latency after reply

Aperta
#2,682 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area/core/memory bug
Lingua principale
Java
Stelle
5.6k
Fork
1.3k
Merge medio
4g 12h
PR unite (30g)
77

Descrizione

### Description
After the agent finishes streaming its reply, the event stream stays open for several seconds. SSE/HTTP consumers keep the "generating" state until the stream completes. Root cause: `MemoryFlushMiddleware.onAgent` appends a **synchronous LLM extraction call** to the agent event stream via `.concatWith()`:

```java
return next.apply(input)
.concatWith(
Mono.defer(() -> doFlush(agent, rc))
.subscribeOn(Schedulers.boundedElastic())
...
```

`concatWith` semantics require the appended `flushMemories` (an extra `model.stream` call in `MemoryFlushManager`) to finish before the main stream completes. `subscribeOn(boundedElastic)` only moves the work to another thread — it does not shorten the completion wait.

Secondary impact: `HarnessGateway.withGatedStream` serialises turns per `gateKey`; while a flush is pending, the next user message in the same session is blocked until it completes.

### Repro
- Default `MemoryConfig.FlushTrigger.always()` (flush after every call).
- Observe the gap between the last text event and stream `onComplete` / SSE connection close ≈ duration of one extra LLM round-trip.

### Expected
The event stream should complete immediately after the last agent event; memory extraction should run in the background (e.g. fire-and-forget on a daemon executor, like `SkillCuratorMiddleware`), with pending flushes drained on `HarnessAgent.close()` so no memory is lost on shutdown.

### Related
- `agentscope-harness/.../middleware/MemoryFlushMiddleware.java`
- `agentscope-harness/.../memory/MemoryFlushManager.java` (`flushMemories`)
- `MemoryMaintenanceMiddleware` has the same pattern (and a production `.block()`) — may warrant a follow-up.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.