[BUG] Prevent concurrent chat sends from corrupting session ordering
@vizsatiz is already working on this.
Since Sep 18, 2026.
- Dominant language
- Python
- Stars
- 200
- Forks
- 30
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 35
Description
Describe the bug
Concurrent requests to POST /v1/chat-sessions/{session_id}/messages are not coordinated per session. Each request commits its user message, loads history, invokes the LLM, and later commits its assistant response independently. If two sends overlap, provider completion order can differ from request order, producing a conversation such as:
user A
user B
assistant response to B
assistant response to A
Each response may also be generated from a different snapshot of the history, so the model can answer with inconsistent context. The current timestamp-based ordering does not correlate an assistant response with the user turn that triggered it, and retries can further amplify the problem.
Related implementation: send_message.
Related pull request: #383
To Reproduce
Steps to reproduce the behavior:
- Create an enabled chatbot and a chat session.
- Send two messages to the same session concurrently, for example with two clients or overlapping requests.
- Use a provider or test double that delays the first request so the second request completes first.
- Inspect the persisted messages and the histories sent to the provider.
A minimal reproduction should assert that overlapping sends can persist assistant replies in completion order rather than turn order.
Expected behavior
A session should serialize or coordinate concurrent sends so that each assistant response is associated with the correct user turn and the model receives a consistent history. Retrying a request should not create duplicate user turns or duplicate assistant responses.
Possible implementations include a per-session turn queue/lock, optimistic session-version checks, a turn identifier or monotonic sequence, and an idempotency key for client retries. The solution should preserve the existing behavior where the user's message survives an inference failure.
Screenshots
Environment (please complete the following information):
- python version:
- OS:
- flo-ai version:
Additional context
The current design commits the user message before inference, which is useful for provider failures, but separate transactions alone do not prevent two requests from generating against conflicting histories. Timestamp-only ordering also cannot reliably establish semantic turn order under concurrency. Add regression tests covering overlapping JSON sends, overlapping SSE sends, provider completion in reverse order, and client retries after a timeout or disconnect.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.