MoonshotAI / MoonshotAI/kimi-cli

会话持久化缺少 `fsync`,异常退出时存在丢数据风险 || Session persistence lacks `fsync`, and there is a risk of data loss when exiting abnormally.

Open
#2,093 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
11.4k
Forks
1.3k
Avg merge
9h 47m
Merged PRs (30d)
2

Description

What feature would you like to see?

问题描述

目前 context.jsonlwire.jsonl 都是通过 aiofiles.open(..., "a") 追加写入的,但代码在写入后没有显式 flush 操作系统缓存,也没有调用 fsync() 强制落盘。这意味着如果终端被异常关闭(例如直接点关闭窗口、SSH 断开、kill -9 等),那些已经"写入"的数据可能仍停留在操作系统页缓存中,最终无法到达磁盘。

实际使用中会发现:一个长时间运行的会话看起来只有在 CLI 正常退出时才"保存"(塞水)。如果进程在某一轮对话处理到一半时被终止,当前轮次的消息可能会完全丢失。

涉及代码

  1. kimi_cli/soul/context.py —— append_message()update_token_count()checkpoint()

    async with aiofiles.open(self._file_backend, "a", encoding="utf-8") as f:
        await f.write(message.model_dump_json(exclude_none=True) + "\n")
    
  2. kimi_cli/wire/file.py —— append_record()

    async with aiofiles.open(self.path, mode="a", encoding="utf-8") as f:
        ...
        await f.write(_dump_line(record))
    

建议修复

在每个关键持久化路径的 write() 之后,显式 flush 并 fsync:

await f.flush()
await asyncio.to_thread(os.fsync, f.fileno())

这些写入发生在消息粒度(而非 token 粒度),因此 fsync 的性能开销在典型的 CLI 使用场景下可以忽略不计,同时能显著提升数据可靠性。

其他方案对比

  • 定时 fsync:例如每 N 秒或每 M 条消息同步一次。这样可以减少系统调用次数,但仍会留下数据丢失的时间窗口。
  • 行缓冲模式:Python 的 open(..., buffering=1) 仅在文本模式下按行缓冲,仍然不能保证数据持久化到磁盘,仍需配合 fsync

考虑到对话消息的频率本身就较低,每条消息后执行 fsync 是最简单且最安全的做法。

Additional information

No response


What feature would you like to see?

Problem description

Currently, context.jsonl and wire.jsonl are appended through aiofiles.open(..., "a"), but the code does not explicitly flush the operating system cache after writing, nor does it call fsync() to force disk flushing. This means that if the terminal is closed abnormally (such as closing the window directly, SSH disconnecting, kill -9, etc.), the data that has been "written" may still stay in the operating system page cache and eventually fail to reach the disk.

In actual use, you will find that a long-running session seems to be "saved" (stuffed) only when the CLI exits normally. If the process is terminated in the middle of a conversation round, the current round of messages may be completely lost.

Involving code

  1. kimi_cli/soul/context.py —— append_message(), update_token_count(), checkpoint()

    async with aiofiles.open(self._file_backend, "a", encoding="utf-8") as f:
        await f.write(message.model_dump_json(exclude_none=True) + "\n")
    
  2. kimi_cli/wire/file.py —— append_record()

    async with aiofiles.open(self.path, mode="a", encoding="utf-8") as f:
        ...
        await f.write(_dump_line(record))
    

Suggested fixes

Explicitly flush and fsync after each write() of the critical persistence path:

await f.flush()
await asyncio.to_thread(os.fsync, f.fileno())

These writes occur at message granularity (rather than token granularity), so the performance overhead of fsync is negligible in typical CLI usage scenarios, while significantly improving data reliability.

Comparison of other solutions

  • Scheduled fsync: For example, synchronize every N seconds or every M messages. This reduces the number of system calls but still leaves a window of time for data loss.
  • Line buffering mode: Python's open(..., buffering=1) only buffers by row in text mode. It still cannot guarantee that the data is persisted to the disk, and fsync still needs to be used.

Considering that the frequency of conversation messages is inherently low, executing fsync after each message is the simplest and safest approach.

Additional information

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading kimi_cli/soul/context.py methods append_message(), update_token_count(), and checkpoint(), then kimi_cli/wire/file.py's append_record(). Check how aiofiles exposes flushing and file descriptors, and look for existing persistence tests. Done means writes in the named paths are flushed and synced as requested, with tests covering the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.