utopia-php / utopia-php/monorepo

Codec\Igbinary preserves objects where Codec\Json flattens them, and nothing tells the publisher

Open
#307 0 comments 0 reactions 1 assignee View on GitHub

@levivannoort is already working on this.

Since Sep 19, 2026.

Dominant language
PHP
Stars
3
Forks
4
Avg merge
12h 25m
Merged PRs (30d)
103

Description

Codec\Igbinary::encode() is a bare igbinary_serialize(). Codec\Json goes through json_encode + json_decode(assoc: true), which flattens every object to an array. So the two codecs do not produce equivalent values, and the difference is invisible until a handler receives a type it has never been given.

That flattening is not incidental — it is the de-facto contract every handler has been written against, because JSON has been the only writer.

What it cost

Staging flipped _APP_QUEUE_CODEC to igbinary. worker-webhooks wedged within minutes:

Utopia\Database\Document::__construct(): Argument #1 ($input) must be of type array,
Utopia\Database\Document given

A webhooks payload carries a Document. Under JSON the handler had always received an array. 701 messages, each burning its full redelivery budget while holding a maxAckPending slot; the queue backed up to 3,434 and delivered nothing for hours. (#306 fixes the retry half of that. This issue is the half that would have stopped it happening at all.)

Why detection wasn't enough

The obvious check — sample the queue and compare — cannot work. The wire is JSON, so every object on it has already been flattened. A sweep of 513 envelopes across six queues reported clean, and webhooks was the seventh.

The only place the publisher's own value is visible is encode().

The design tension

An encode() that walks the value graph and refuses objects would catch this at the publisher, immediately, in the process that owns the payload. But a recursive PHP walk on every publish taxes exactly the hot path igbinary exists to make fast — igbinary_serialize is C, the walk would not be, and queue publishes sit on the request path.

Options, none obviously right:

  1. Walk and throw on every encode. Correct and loud, unmeasured cost. Payloads are small and profiling (appwrite-labs/cloud#5895) puts encode well below broker round-trips in the per-message budget, so this may be cheap enough — but that is a guess until measured.
  2. Walk only under a flag, as an audit run before a codec change. This is what ReportingCodec does in appwrite-labs/cloud#5965; it belongs in this package if it is the answer, so every consumer gets it rather than one.
  3. Normalise instead of refusing — round-trip through array form so igbinary matches JSON exactly. Safe, and destroys the performance argument for igbinary entirely.
  4. Document and leave it, on the grounds that the audit flag plus #306 makes the failure survivable rather than preventable.

I lean 1 if a benchmark says the walk is noise against a publish, and 2 otherwise. What I am not prepared to do is ship 1 on the assumption.

What would settle it

Benchmark igbinary_serialize($payload) against walk($payload) + igbinary_serialize($payload) on representative envelopes. If the walk is within noise of the publish round-trip, take option 1.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.