Stack overflow in binary format writers (CBOR/MessagePack/UBJSON/BSON) due to unbounded recursion

Summary

The binary format writers (binary_writer.hpp) recurse once per nesting level with no depth limit, so serializing a deeply nested value crashes the process. This is the counterpart of #5104, which covers the binary readers; both directions are affected, and neither is fixed.

The text parser is iterative, so json::parse accepts input of any depth. A value that the library happily builds can therefore crash it on the way back out.

Affected functions

• json::to_cbor()
• json::to_msgpack()
• json::to_ubjson() (and BJData)
• json::to_bson()

Reproduction

Against develop (734fd305a), on macOS/arm64 with the default 8 MB stack:

#include <nlohmann/json.hpp>
#include <string>

int main()
{
    const std::size_t depth = 200000;
    const nlohmann::json j = nlohmann::json::parse(
        std::string(depth, '[') + "0" + std::string(depth, ']'));

    const auto v = nlohmann::json::to_cbor(j);   // SIGSEGV
    return static_cast<int>(v.size());
}

The depth needed varies with build and stack size, as usual for stack exhaustion; CBOR and MessagePack survive depth 100,000 at -O2 on this machine and fail at 200,000. BSON is measured with a nested object, since BSON requires an object at the root.

Root cause

Each writer calls itself for every element:

• CBOR: write_cbor() (binary_writer.hpp:96) → binary_writer.hpp:303 (array), :405/:406 (object)
• MessagePack: write_msgpack() (:420) → :614 (array), :727/:728 (object)
• UBJSON/BJData: write_ubjson() (:747) → :842 (array), :944 (object)
• BSON: write_bson_element() (:1238) → write_bson_object() (:1302) / write_bson_array() (:1160) → write_bson_element() (:1306/:1166)

BSON recurses twice over the same value. Besides writing, it computes each document's length up front through a second, independent recursive walk: calc_bson_element_size() (:1190) → calc_bson_object_size() (:1285) / calc_bson_array_size() (:1133) → calc_bson_element_size() (:1290/:1139). Both walks need bounding, which makes BSON the most involved of the four.

Suggested fix

The same shape used for the destructor in #1436 and for copying, serializing and comparing in #5389 / #5285 / #5390: descend a bounded number of levels, then finish the rest on an explicit stack. That keeps the fast path for ordinary values and removes the limit on depth, without a new exception or a new macro.

CBOR, MessagePack and UBJSON are structurally similar and could share an approach. BSON's size pass is separate work and may be worth its own change.

Related

• #5104 — the same problem in the binary readers
• #5387 — the same problem in the copy constructor and dump()
• #1436 — where the destructor was fixed this way

Written by Claude Code.

Start in binary_writer.hpp with write_cbor(), write_msgpack(), write_ubjson(), and the BSON writing and size-calculation functions listed in the issue. Review the bounded-depth approach referenced from #1436 and the related serialization issues before deciding how to use an explicit stack. Done means deeply nested values serialize without stack exhaustion, including both BSON traversal passes.