oceanbase / oceanbase/powercontext

bug: Memory append storage and latency grow superlinearly with entry history

Open
#1,321 2 comments 0 reactions 1 assignee View on GitHub

@MaoMengww is already working on this.

Since Aug 23, 2026.

bug
Dominant language
Python
Stars
1.1k
Forks
212
Avg merge
1d 11h
Merged PRs (30d)
199

Description

Describe the bug

Repeated explicit Memory appends have superlinear persistent-storage growth and steadily increasing write latency, even when each entry contains only a few bytes of text.

Every Memory Revision stores the complete current manifest. Appending one entry therefore writes manifests of size 1, 2, 3, ... N across the immutable revision history. Entry bodies are not duplicated, but the cumulative manifest directory approaches quadratic growth.

There is currently no maximum entry count, manifest-byte budget, automatic Memory split, or tombstone compaction boundary. The unpaginated list_memory_entries path also materializes the complete current manifest and all referenced entry versions.

The Memory RFC identifies split and compaction thresholds as future work, but the current service can grow past practical limits without an explicit error or degradation signal.

Steps to reproduce

At commit a6e0dfe8b0b20b9a40e905aaa07c62427c1418f4:

  1. Create a fresh SQLite-backed Server with Dashboard, MCP, metrics, and access logging disabled.
  2. Repeatedly call POST /v1/memory/remember in one scope with one short unique fact per request.
  3. Close the Server.
  4. Run PRAGMA wal_checkpoint(TRUNCATE) and VACUUM.
  5. Record the database size and average duration of the final 20 writes.
  6. Repeat with 50, 100, and 200 entries.

Observed:

Entries SQLite database bytes Average final 20 writes
50 671,744 40.57 ms
100 1,634,304 65.21 ms
200 5,222,400 111.81 ms

The test body was only bounded fact NNNN.

Relevant paths:

  • src/powercontext/builtin/artifacts/memory/models.py:62-65
  • src/powercontext/builtin/artifacts/memory/service.py:699-731
  • src/powercontext/builtin/persistence/artifacts.py:243-262
  • src/powercontext/builtin/persistence/memory.py:157-175
  • docs/en/rfcs/0014_memory_layer_design.md, especially the deferred split/compaction thresholds
Expected behavior

A long-lived Memory service needs an explicit, observable capacity contract.

At minimum:

  • Enforce a current-manifest entry or byte budget before unbounded growth.
  • Return a stable error or route new entries to another Memory when the budget is reached.
  • Paginate or otherwise bound list_memory_entries and complete-history operations.
  • Define and test split and inactive-tombstone compaction behavior.
  • Add a scale regression test that measures storage and write amplification at increasing entry counts.

The exact split/compaction design may require an RFC, but the current API should not accept growth indefinitely without a budget.

Actual behavior

Each append reconstructs and persists the complete manifest in a new immutable Artifact Revision. Storage and write work increase with the number of prior entries, with no configured ceiling or automatic split.

Environment
  • PowerContext: 0.0.3.dev11+ga6e0dfe8b.d20260823
  • Commit: a6e0dfe8b0b20b9a40e905aaa07c62427c1418f4
  • Python: 3.14.3
  • Backend: SQLite with WAL, checkpointed and vacuumed before measurement
  • OS: Windows 11
  • External inference and embedding providers: disabled
Are you willing to submit a PR to fix this bug?
  • Yes, I would like to submit a PR.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.