obs: byte-aware request chunking for content-capable sinks (Datadog 1 MB/log + 5 MB/request; SLS parity)

Problem

The content-capable observability sinks (datadog and aliyun_sls) build one HTTP request per batch and POST it whole. They declare max_batch_bytes: None and rely on the pipeline's count-based batch cap — they do not self-enforce the upstream's byte limits, even though pipeline.rs assigns per-request byte chunking to the sink ("the sink owns wire encoding, including chunking a batch down to its own per-request byte limit inside append_batch").

Datadog's logs intake limits (https://docs.datadoghq.com/api/latest/logs/#send-logs):

• ≤ 1000 logs per request
• ≤ 5 MB uncompressed per request
• ≤ 1 MB per single log

Exposure (opt-in content_mode = full only)

The default metadata_only path is safe — each log is ~1–2 KB, so a count-capped batch stays far under every limit. Under full:

• (A) per-log 1 MB — a single log carries both gen_ai.prompt and gen_ai.completion (each capped by content_max_bytes, max 1 MiB) plus metadata, so one log can exceed Datadog's 1 MB/log limit.
• (B) per-request 5 MB — even at the default 128 KiB/field cap, a batch of ~17+ full logs (fewer at a higher content_max_bytes) can exceed 5 MB/request.

When either is hit, Datadog rejects the whole request (400/413) → classified Permanent → the batch is dropped. This is operator-visible via SinkStats::last_error (an HTTP 413/400 delivery error), not silently swallowed — but the records are lost.

Fix

Byte-aware request splitting, shared across the content-capable sinks (datadog + aliyun_sls have the identical shape — implement once as a shared helper, not per-sink):

1. Split a batch's records into requests each ≤ 5 MB uncompressed and ≤ 1000 logs.
2. Truncate any single log whose content alone exceeds the per-log limit (set the existing content_truncated marker).
3. Then capabilities() can honestly declare max_batch_bytes.

Tests

• A record at the max content cap encodes to a single log < 1_000_000 bytes.
• A batch that would exceed 5 MB is split into multiple POSTs (assert N requests at the mock intake).
• A > 1000-record batch is split.

Not blocking #550

The DP datadog sink (#550) ships with this deferred by agreement: the default path is safe, the exposure is opt-in + operator-visible, and this is shared sink/pipeline infra (SLS carries the identical limitation today). Surfaced by the §8 audit of #550. Refs #688, #57.

Start in pipeline.rs and inspect the append_batch implementations for the datadog and aliyun_sls sinks, including how they encode records and expose capabilities(). Then review the mock intake tests and add coverage for per-log size, 5 MB request splitting, and batches over 1000 records. Done means both sinks share byte-aware splitting, truncate oversized logs with content_truncated, and declare their request byte limit.