Codex can mark generated binary artifacts as validated despite format invariants being violated (MOBI6/Kindle repro)

Summary

Codex generated a custom MOBI6/PalmDOC file, reported that it had been validated, and then repeated that claim after a first repair. On a physical Kindle, page turning at a particular CJK text boundary froze the reader. The validation checked parsing and UTF-8 decoding but missed required PalmDOC record invariants.

This is best treated as a model-behavior / artifact-validation gap rather than a request for first-party MOBI conversion support: when Codex writes a binary format itself, it should not claim device compatibility unless it has verified the format's hard invariants and clearly separated structural checks from real-device testing.

Environment

• Surface: ChatGPT Work / Codex web
• Output format: legacy MOBI6 / PalmDOC
• Target device: Kindle 11th generation
• Firmware: 5.18.3
• Content characteristic: long UTF-8 CJK text
• No copyrighted file is attached to this report.

What happened

Initial output

The custom writer divided the uncompressed UTF-8 HTML into nominal 4096-byte records but did not add the PalmDOC UTF-8 boundary-overlap trailer. Some records therefore ended inside a three-byte Chinese character.

The file was reported as validated, but the physical Kindle froze when turning the page at the affected boundary.

A concrete failing raw boundary was byte offset 102400, which split a three-byte CJK code point.

First attempted repair

Codex changed the logical record sizes to 4094/4095 bytes so that each record independently decoded as UTF-8. It again reported the MOBI as validated after KindleUnpack and per-record UTF-8 checks.

However, that output still violated the fixed 4096-byte PalmDOC text-block invariant:

• decoded text length: 15,922,004 bytes
• expected record count: ceil(15,922,004 / 4096) = 3888
• actual text records: 3889
• record-size distribution:
  • 1530 × 4096 bytes
  • 1197 × 4094 bytes
  • 1161 × 4095 bytes
  • final record: 311 bytes

The Kindle still froze at page turn.

Corrected output

The working repair used:

• fixed 4096-byte uncompressed logical text blocks
• PalmDOC bit 0 set in trailing-data flags
• UTF-8 boundary-overlap bytes appended after each compressed record when the logical block ends inside a multibyte code point
• consistent last_content_record, FLIS/FCIS indices, locale, and EXTH language metadata

For the corrected build:

• text record count matched ceil(text_length / 4096)
• all non-final logical text records decoded to exactly 4096 bytes
• every overlap trailer matched the prefix of the next logical record
• KindleUnpack succeeded
• the reconstructed raw HTML decoded strictly as UTF-8

Why the existing validation was insufficient

The checks answered only:

1. Can a desktop parser unpack this file?
2. Can each chosen record payload be decoded as UTF-8?

They did not answer:

1. Are PalmDOC logical text blocks exactly 4096 bytes except the final block?
2. Does text_record_count == ceil(text_length / 4096)?
3. If a UTF-8 sequence crosses a boundary, is the required overlap trailer present and advertised by the correct trailing-data flag?
4. Do content and auxiliary-record indices agree?
5. Has compatibility been verified on a physical Kindle, or only inferred?

Because the output passed the weaker tests, Codex presented a stronger conclusion than the evidence supported.

Expected behavior

When generating a binary artifact with a custom writer, Codex should:

• identify and test hard format invariants, not only successful parse/decode;
• avoid describing an output as “Kindle-compatible” solely because KindleUnpack accepts it;
• distinguish “structurally validated” from “tested on a real device”;
• treat a user-reported device freeze as evidence that the validation oracle is incomplete;
• after a failed repair, compare the writer against the format’s record model instead of optimizing only for the previously observed symptom.

Suggested regression evaluation

Use a synthetic, non-copyrighted UTF-8 CJK stream deliberately constructed so a three-byte character crosses every possible position near a 4096-byte boundary. Ask the model to write MOBI6/PalmDOC and validate that:

• text_record_count == ceil(text_length / 4096);
• all non-final logical text blocks are exactly 4096 bytes;
• trailing-data flags match the actual trailers;
• overlap length and bytes match the next record prefix;
• reconstructed text equals the original byte-for-byte;
• last_content_record, FLIS, and FCIS positions are internally consistent;
• any compatibility statement explicitly says whether a physical Kindle was tested.

Privacy

This report intentionally contains no book text, source file, generated ebook, account details, or other copyrighted attachment.

No repository file, test, or entry point is named; start by locating the artifact-generation and validation path, then inspect how KindleUnpack and per-record UTF-8 checks are used. Use the suggested synthetic CJK stream to exercise the listed PalmDOC invariants, and consider the work done when those checks pass and compatibility claims distinguish structural validation from physical-device testing.