MoonshotAI / MoonshotAI/kimi-code

minidb query-store fails permanently on filesystems without hardlink support (link() EPERM), breaking kimi web

Open
#3,535 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
7.5k
Forks
1.2k
Avg merge
11h 53m
Merged PRs (30d)
350

Description

Environment

  • OS: HarmonyOS (HongMeng Kernel 1.13.0, aarch64) — filesystem is hmdfs/tmpfs under /storage, which does not support hard links: link(2) always fails with EPERM
  • Node: v26.7.0
  • kimi-code: 0.40.1 (worked fine on 0.39.1)

Symptom

After upgrading to 0.40.1, kimi web starts and serves the UI, but agent turns stall mid-stream: subagent output freezes, quota/plugin info in the UI stops refreshing, and no requests reach the API. The TUI keeps working in the same environment.
The log is flooded (~20 entries/sec, 120k+ occurrences) with:

WARN failed to flush session index mirror chunk pending=1 failures=120918
  error="Error: EPERM: operation not permitted, link
  '.../cache/query-store/cluster.meta.json.tmp-64164' ->
  '.../cache/query-store/cluster.meta.json'"
WARN session index mirror giving up until the next record; reconciliation will heal
WARN session index read model degraded; serving authoritative reads reason="prepare failed"

Root cause

The new minidb query-store uses fs.promises.link(tmp, path) for atomic create-if-absent in several places and only tolerates EEXIST:

  • Topology.open (cluster/utils): writeFile(tmp) + link(tmp, cluster.meta.json) — throws on EPERM, so the cluster meta file is never created and every open attempt fails forever (even after the file exists, since link() fails with EPERM rather than EEXIST on such filesystems).
  • Lockfile tryCreate and snapshot generation use the same pattern.
    On filesystems without hardlink support this makes the whole query-store unusable, floods the log, and wedges the web server's event/session pipeline.

Suggested fix

Fall back when link() fails with EPERM (or ENOSYS), e.g.:

try {
  await fs.promises.link(tmp, dst);
} catch (e) {
  if (e.code === 'EPERM' || e.code === 'ENOSYS') {
    // copyFile with COPYFILE_EXCL gives the same atomic create-if-absent
    // semantics and throws EEXIST when the destination exists
    await fs.promises.copyFile(tmp, dst, fs.constants.COPYFILE_EXCL);
  } else throw e;
}

Workaround confirmed working

Preloading a small Node shim (NODE_OPTIONS=--import ...) that patches fs.promises.link with the copyFile fallback above fixes 0.40.1 completely on this machine: the query-store initializes, the log flood stops, and kimi web works normally. Setting KIMI_CODE_EXPERIMENTAL_PERSISTENCE_MINIDB_READMODEL=0 also avoids the issue by disabling the feature.
(That's how kimi code fix itself anyway.)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the minidb query-store paths mentioned in Topology.open, the lockfile tryCreate logic, and snapshot generation, focusing on their fs.promises.link calls. Reproduce the EPERM behavior on a filesystem without hardlink support, then verify that query-store initialization, locking, snapshots, and kimi web's session pipeline continue working without repeated failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.