agentscope-ai / agentscope-ai/agentscope-java

MemoryFlushMiddleware delays agent event stream completion, causing perceived latency after reply

Ouverte
#2,682 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
area/core/memory bug
Langage dominant
Java
Étoiles
5.6k
Forks
1.3k
Merge moyen
4 j 12 h
PR mergées (30 j)
77

Description

### Description
After the agent finishes streaming its reply, the event stream stays open for several seconds. SSE/HTTP consumers keep the "generating" state until the stream completes. Root cause: `MemoryFlushMiddleware.onAgent` appends a **synchronous LLM extraction call** to the agent event stream via `.concatWith()`:

```java
return next.apply(input)
.concatWith(
Mono.defer(() -> doFlush(agent, rc))
.subscribeOn(Schedulers.boundedElastic())
...
```

`concatWith` semantics require the appended `flushMemories` (an extra `model.stream` call in `MemoryFlushManager`) to finish before the main stream completes. `subscribeOn(boundedElastic)` only moves the work to another thread — it does not shorten the completion wait.

Secondary impact: `HarnessGateway.withGatedStream` serialises turns per `gateKey`; while a flush is pending, the next user message in the same session is blocked until it completes.

### Repro
- Default `MemoryConfig.FlushTrigger.always()` (flush after every call).
- Observe the gap between the last text event and stream `onComplete` / SSE connection close ≈ duration of one extra LLM round-trip.

### Expected
The event stream should complete immediately after the last agent event; memory extraction should run in the background (e.g. fire-and-forget on a daemon executor, like `SkillCuratorMiddleware`), with pending flushes drained on `HarnessAgent.close()` so no memory is lost on shutdown.

### Related
- `agentscope-harness/.../middleware/MemoryFlushMiddleware.java`
- `agentscope-harness/.../memory/MemoryFlushManager.java` (`flushMemories`)
- `MemoryMaintenanceMiddleware` has the same pattern (and a production `.block()`) — may warrant a follow-up.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.