anomalyco / anomalyco/opencode

server: V1 migration dies with RangeError: Out of memory on a single oversized part row

Open
#50,076 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Summary

The V1 -> V2 data migration aborts with RangeError: Out of memory when a single
part row is large enough that materializing and re-serializing it exceeds the
heap. One oversized row halts the entire migration, leaving the remaining
sessions permanently unmigrated until the row is manually repaired. The
migration is cursor-based and resumable, but it retries the same row on every
service start, so it never makes progress on its own.

I searched existing issues. This looks distinct from the other V1 migration
reports: #41739 is a slow full scan of a ~28 GiB database with no OOM on an old
next build, and #45829 / #41869 / #41346 are SQLite syntax errors. Here the
database migrates fine except for one row whose size alone exhausts the heap.

Environment

  • opencode version: 2.0.10
  • OS: Darwin 27.0.0 (darwin arm64), macOS
  • Terminal: TERM_PROGRAM=iTerm.app, TERM=xterm-256color, COLORTERM=truecolor
  • Shell: /bin/zsh
  • Install/channel: latest (curl installer, ~/.opencode/bin/opencode)
  • Active plugins: 5 configured local plugins (4 are V1-era and fail to load on
    V2, which is expected per the migration guide; unrelated to this bug)

Reproduction

  1. On V1, run a read tool call against a large PDF so the tool part persists
    the file inline as a base64 data: URL in part.data. In this case the
    resulting row was 233,295,050 bytes (233 MB) in a single part.data value.
  2. Upgrade to opencode 2.0.10 and let the background service start.
  3. Observe the V2 schema migrations all succeed, then the V1 data migration
    begins walking sessions in descending session-ID order.
  4. When it reaches the session containing the oversized part, the migration
    dies.

Expected Behavior

An individual oversized or unparseable row should not be able to halt the whole
migration. I would expect either streaming/chunked handling of large part
payloads, or the same Skipped V1 migration row treatment that is already
applied to rows the migrator considers invalid, so the migration can continue
and report what it skipped.

Actual Behavior

The migration aborts and logs a single line:

timestamp=2026-09-20T01:00:29.330Z level=ERROR run=0b69438b
message="V1 migration failed" cause="Cause([Die(RangeError: Out of memory)])" role=server

State at that point:

  • session (V1): 409 rows, session_v2: 398 migrated
  • message (V1): 26,120 rows, session_message: 24,631 migrated
  • Resume cursor in kv:
    migration.v1-v2 = {"phase":"sessions","cursor":"ses_002133f23..."}

11 sessions never migrated. Restarting the service resumes from the cursor,
immediately hits the same row, and dies again, so the failure is permanent
without manual intervention.

The error message does not identify the session, message, or part responsible,
which made this fairly slow to diagnose against a 6 GB database. Naming the row
being processed when the migration dies would have made this immediate.

Additional Context

Offending payload shape. The 233 MB row was a single read tool part whose
state.attachments[0].url held an entire PDF inline:

{"type":"tool","tool":"read","state":{"status":"completed",
 "input":{"filePath":".../slides.pdf"},"output":"PDF read successfully",
 "attachments":[{"type":"file","mime":"application/pdf",
   "url":"data:application/pdf;base64,JVBERi0xLjMK...   <- 233 MB

That one row was 93% of its session's total part bytes. As a JS string it is
roughly 466 MB in UTF-16 before any parse or re-serialize, which is consistent
with the OOM.

Workaround. After backing up the database, I replaced just the base64
payload with a short placeholder, keeping the JSON shape and a syntactically
valid data: URL, and the migration resumed and completed. Interestingly the
migrator then rejected the repaired row as invalid-part:

level=WARN message="Skipped V1 migration row" reason=invalid-part
  sessionID=ses_... messageID=msg_... partID=prt_... observedType=tool

So the skip path this row needed already exists; it just is not reached when the
failure mode is memory exhaustion during row materialization.

Secondary observation, possibly a separate issue. Migrated sessions whose
directory is not a VCS repository were all assigned project_id = 'global'
(210 of 409 here, spanning 12 directories), while sessions in git repositories
received real project IDs. V2 then created a new project row for the same
non-git directory on first use. The result is that opencode session list, which
scopes to the current project, showed 1 session in a directory that has 90. The
data is intact and GET /api/session does return the global ones, so this is a
grouping/visibility wart rather than data loss, but the split between a migrated
global binding and a freshly created project row for the identical directory
looks unintended. Happy to file separately if preferred.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the V1-to-V2 cursor migration that materializes and re-serializes session, message, and part rows, then trace how it reaches the existing invalid-part skip path. Reproduce with a single oversized part.data value like the reported 233 MB payload. Done means one oversized row no longer aborts or blocks later sessions, the row is reported with identifying context, and migration progress remains resumable.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.