agentscope-ai / agentscope-ai/agentscope-java
MemoryFlushMiddleware delays agent event stream completion, causing perceived latency after reply
- Lingua principale
- Java
- Stelle
- 5.6k
- Fork
- 1.3k
- Merge medio
- 4g 12h
- PR unite (30g)
- 77
Descrizione
### Description
After the agent finishes streaming its reply, the event stream stays open for several seconds. SSE/HTTP consumers keep the "generating" state until the stream completes. Root cause: `MemoryFlushMiddleware.onAgent` appends a **synchronous LLM extraction call** to the agent event stream via `.concatWith()`:
```java
return next.apply(input)
.concatWith(
Mono.defer(() -> doFlush(agent, rc))
.subscribeOn(Schedulers.boundedElastic())
...
```
`concatWith` semantics require the appended `flushMemories` (an extra `model.stream` call in `MemoryFlushManager`) to finish before the main stream completes. `subscribeOn(boundedElastic)` only moves the work to another thread — it does not shorten the completion wait.
Secondary impact: `HarnessGateway.withGatedStream` serialises turns per `gateKey`; while a flush is pending, the next user message in the same session is blocked until it completes.
### Repro
- Default `MemoryConfig.FlushTrigger.always()` (flush after every call).
- Observe the gap between the last text event and stream `onComplete` / SSE connection close ≈ duration of one extra LLM round-trip.
### Expected
The event stream should complete immediately after the last agent event; memory extraction should run in the background (e.g. fire-and-forget on a daemon executor, like `SkillCuratorMiddleware`), with pending flushes drained on `HarnessAgent.close()` so no memory is lost on shutdown.
### Related
- `agentscope-harness/.../middleware/MemoryFlushMiddleware.java`
- `agentscope-harness/.../memory/MemoryFlushManager.java` (`flushMemories`)
- `MemoryMaintenanceMiddleware` has the same pattern (and a production `.block()`) — may warrant a follow-up.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.