iOfficeAI / iOfficeAI/AionCore

[Bug]: Lark channel - Feishu retries every event (ack not accepted); retried messages re-executed as new chats

Open
#796 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
105
Forks
169
Avg merge
5h 58m
Merged PRs (30d)
84

Description

# [Bug] Lark channel: platform marks every long-connection event push as FAIL/timeout and retries it; redeliveries are re-executed as new chat messages

### Platform

Windows 11

### AionUi Version

2.1.49 (bundled AionCore ≈ v0.1.59). Checked the v2.1.50 changelog — no related fix.

### Bug Description

Every message a user sends to the Lark (Feishu) bot over the WebSocket long connection is processed by AionUi **3–4 times** over the following ~6 hours. From the user's side it looks like the bot "re-asks / re-answers questions that were already asked".

Root cause appears to be two compounding problems:

1. **The platform never accepts AionUi's event acknowledgment.** In the Feishu Open Platform event log, every push (including the original, first delivery) is recorded with status **FAIL** after `costMills` ≈ 3752 ms (a timeout). Feishu therefore retries each event on its failure-retry schedule (~5 min / ~1 h / ~6 h, max 3 retries), reusing **the same `eventId`**.
2. **AionUi's dedup window is far shorter than the retry window.** `LARK_EVENT_DEDUP_TTL` is only 5 minutes (`crates/aionui-channel/src/constants.rs:55`) and the cache is in-memory. The 1-hour and 6-hour redeliveries (and even the 5-minute one, which lands slightly past the TTL) all pass through and are dispatched to the agent again as brand-new user messages.

Each redelivery triggers a full agent run (token cost) and the bot sends the answer card to the user again.

### Evidence

**Feishu Open Platform event log (filtered 返回状态 = FAIL):** the same `eventId` appears multiple times at retry-schedule intervals, all FAIL with `costMills` ≈ 3752 ms and a `timeout` field, e.g.:

- `eventId …52911e47310330…` pushed at 2026-08-06 20:02:5x and again at 21:02:59.731 (same event, ~1 h apart)
- `eventId …205e3c97d977…` pushed at 2026-08-06 20:01:15.722 and again at 2026-08-07 02:01:2x (~6 h apart, to the second)
- `eventId …4a545e5cd5db…4944` pushed at 2026-08-06 20:00:21.860 and again at 2026-08-07 02:00:26.774

**AionUi local message store** — each redelivery is stored and executed as a new user message (all times local, UTC+8):

| User message (sent once) | First processed | Re-dispatched at |
|---|---|---|
| "Hi?" | 08-06 19:13:39 | 20:19:02, 08-07 02:19:06 |
| "Which model are you?" | 19:45:14 | 19:51:05 (failed: conversation busy), 20:51:11, 02:51:15 |
| "Translate: 2026年7月20日 …" (461 chars) | 19:57:21 | 20:02:47, 21:02:58, 03:03:02 |
| "Are you on?" | 20:01:12 | 20:04:27, 20:17:02, 02:01:16, 02:04:31, 02:17:07 |

The 6-hour-later redeliveries land exactly +6h00m0Xs after the original, matching the platform retry schedule — these are unattended replays (03:03 AM local), not the user re-sending.

**aioncore log excerpts:**

```text
[2026-08-06 19:13:21.469] [info] aionui_channel::plugins::lark::plugin: Lark WebSocket connected
[2026-08-06 19:16:56.724] [error] aionui_channel::orchestrator: failed to send to agent error=Message send failed: Conversation is busy: conversation 54c4d720 is already running
[2026-08-06 20:02:47.002] [info] aionui_channel::action: message dispatched to agent session_id=019fd6c7-… chat_id=oc_96c6c15b66c1cd5976bbae3312a6631d text_len=1218
[2026-08-06 21:02:56.017] [info] aionui_channel::action: message dispatched to agent session_id=019fd6c7-… chat_id=oc_96c6c15b66c1cd5976bbae3312a6631d text_len=1218 # same message, 1 h later
```

### Steps to Reproduce

1. Configure the Lark channel (WebSocket long connection), pair a user, bind it to an agent conversation.
2. Send any message to the bot; it is answered normally.
3. In Feishu Open Platform → 事件与回调 → 日志检索, filter 返回状态 = FAIL: the push is logged as FAIL after ~3.7 s, and the same `eventId` reappears ~5 min / ~1 h / ~6 h later.
4. Each redelivery is dispatched to the agent again and the bot replies again.

### Expected Behavior

1. The ack frame sent immediately on receipt (`build_ack_frame`, payload `{"code":200}`, sent in `connect_and_listen` before processing) should make the platform treat the push as successful — no FAIL entries, no retries. Please compare the response frame byte-for-byte against the official `larksuite/oapi-sdk-go` WS client (which headers are echoed — AionUi keeps only `type`/`message_id`/`trace_id` — plus `payload_type`/`payload_encoding`, `biz_rt`, and the exact response payload shape).
2. Defense in depth: even when a redelivery does arrive, it should never be re-executed. The event-id dedup should cover the platform's full retry window — persist seen `event_id`s (or `message_id`s) for at least 24 h and across restarts, instead of a 5-minute in-memory cache.

### Related

- #603 — "don't replay pre-authorization messages as chats" (same class of problem: platform redelivery replayed as a new chat; covers the pre-auth case only)
- #604 — card.action.trigger over the long connection

Image
Image

Image

Image

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in crates/aionui-channel/src/constants.rs at LARK_EVENT_DEDUP_TTL, then trace build_ack_frame and connect_and_listen to compare the acknowledgment with the official larksuite/oapi-sdk-go WebSocket client. Done means Feishu accepts the acknowledgment without retries and repeated event or message IDs are not dispatched again for at least 24 hours or across restarts.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, networking
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.