[Enhancement]: Export OTLP traces to a local file

Affected area

Rust core runtime, Python binding, Node.js binding, Go binding, C FFI, Plugins, Observability or exporters, Documentation or examples

Problem or opportunity

ATIF and ATOF can both be written to a file. OTLP can only be shipped to a collector, so it is the one trajectory output you cannot keep on disk.

Impacted use cases:

1. Evaluation. A harness scores the trace it just produced. It reads the trajectory locally, at scoring time, before any publish step and independent of one. Sourcing that from a trace store means depending on an eventually consistent service, and reading back through a query API that returns its own model.
2. Isolated sandboxes. An agent under test in a container has no route to a host collector without a routable address plus an egress rule per provider. ATIF works there today because it lands in a directory that already gets collected when the task ends.
3. No collector at all. Someone running local evaluations may have no OTLP endpoint and still want OTLP as evidence.

The workaround today is to stand up an OTLP receiver just to catch Relay's export and write it to disk. That costs a port, a readiness handshake, and a network hop to move bytes between two local processes.

Proposed enhancement

Add a local file destination to the OpenTelemetry plugin's trace configuration.

Configuration shape

A new array beside traces, shaped like the existing ATOF sink config:

[[components.config.opentelemetry.file_sinks]]
type = "full"
output_directory = "/var/log/nemo-relay"
format = "json_lines"       # or "proto"
mode = "overwrite"          # or "append"

It shares the projection and batching fields with a trace endpoint (mark_projection, mark_exclude_names, attribute_mappings, promote_*, service_*, instrumentation_scope, resource_attributes, the batch knobs). It omits the four that only a network destination has: endpoint, transport, headers/header_env, and timeout_millis.

This does not change existing configuration files. file_sinks is a new optional array. When it is absent nothing about an existing traces entry changes, and the field is skipped on serialization when empty, so a round-trip through nemo-relay plugins edit is byte-identical.

Endpoints and file sinks compose. N endpoints plus M file sinks produce N+M subscribers joined to the same trace fan-out, each receiving the same projected spans, the way multiple endpoints already work.

Modelling the destination

In the Rust config the destination is an enum:

pub enum TraceDestination {
    Otlp(OtlpEndpointSettings),   // endpoint, transport, headers, header_env, timeout
    File(OtlpFileSinkSettings),   // output_directory, path, format, append
}

That makes the two mutually exclusive by construction.

nemo-relay plugins edit can modify a configuration programmatically, so an endpoint-only option set on a file sink must not be dropped silently. Each surface refuses those options by name:

• Rust: the builders record an inapplicable option, and construction fails with endpoint, headers, transport do not apply to a file sink destination.
• Python: assigning endpoint, transport, or timeout_millis on a config built by OpenTelemetryConfig.file_sink(...) raises ValueError.
• Node: endpoint, transport, timeoutMillis, headers, and headerEnv alongside outputDirectory are rejected at config build.
• C FFI: the file-sink entry point takes no endpoint-only parameters.

On-disk format

json_lines is the default: one OTLP/JSON-encoded ExportTraceServiceRequest per line, UTF-8, \n separated, .jsonl extension. That is the serialization described by the OpenTelemetry Protocol File Exporter specification, and what the OTel Python project's opentelemetry-exporter-otlp-json-file produces.

proto is a second option. It writes each request length-delimited, a big-endian u32 byte count followed by the encoded request, matching the OpenTelemetry Collector file exporter's format: proto layout. It is there for consumers who would rather not pay JSON's size and parse cost.

The conversion comes from opentelemetry-proto, the crate opentelemetry-otlp already uses to build its wire payload. Its group_spans_by_resource_and_scope and SpanData -> Span impls produce the same bytes an endpoint would receive, so a span means the same thing whichever destination it goes to.

Non-goals

Rotation, maximum file size, retention, backup counts, compression, and group_by are out of scope here.

Note: The OTel file exporter specification defines one configuration requirement, a configurable output stream defaulting to stdout, and says nothing about rotation or size limits; its "File storage requirements" section covers encoding only. Rotation, max_megabytes, max_backups, compression, and group_by are OpenTelemetry Collector fileexporter options, from a component whose traces, metrics, and logs signals are alpha.

Runtime contract and binding impact

• Rust core: new TraceDestination enum and OtlpFileSinkSettings on OpenTelemetryConfig, plus a SpanExporter implementation that writes OTLP to a file. Everything above the exporter (projection, id generation, batch processing, resource attributes, shutdown) is shared with the endpoint path. SpanExporter::export returns impl Future, so the trait is not dyn-compatible and destination dispatch is a concrete enum.
• Plugin config: new optional file_sinks array. Static validation accepts a file sink as a destination and reports a malformed one per index, as it already does for endpoints. nemo-relay plugins edit lists file sinks beside trace endpoints.
• Python / Node.js / Go / C FFI: each gains the destination and refuses endpoint-only options on it, as described above.
• Durability and permissions: each export is flushed before it is reported as delivered, so a run that exits between batches leaves a readable prefix instead of an empty file. Output is created with owner-only permissions and confined to output_directory, matching the ATOF and ATIF file sinks, because a trajectory carries prompt and response content.
• No behavior change for any existing configuration, in any binding.

Alternatives considered

• transport = "file" on a trace endpoint. Rejected. endpoint is required and URL-validated, so this makes validation conditional on a sibling field and puts a filesystem path in a field named endpoint. It also cannot express "this destination has no timeout."
• Run an OTLP receiver and write what it catches. The current workaround. It costs a port and a readiness handshake, and in a sandbox it also needs a routable address and an egress rule, to move bytes between two local processes.
• Export to a real trace store and read back. The store may not exist, it is eventually consistent, and its query API returns its own model. It also does not serve scoring, which happens locally before any publish.
• opentelemetry_stdout. Its own documentation says it is for debugging and learning, that the output format is not exhaustive, and that it is subject to change.
• Do nothing. Consumers keep standing up private receivers, each one reimplementing framing and flush semantics.

Acceptance criteria

• A [[components.config.opentelemetry.file_sinks]] entry writes projected spans to a local file in json_lines or proto format.
• Existing configuration files parse and round-trip unchanged. An absent file_sinks array produces byte-identical output through plugins edit.
• Endpoints and file sinks can be configured together and both receive the same spans.
• An endpoint-only option set on a file sink is refused by name on every binding. No silent ignore.
• Output is owner-only, confined to output_directory, and each delivered batch is durable on disk.
• Two file sinks writing the same path are rejected at activation.
• Tests: exporter round-trip decoded by an independent reader (prost and serde_json, not the module's own encoder), OTLP/JSON conformance (camelCase members, hex traceId/spanId), framing, config validation, destination exclusivity, editor schema, and per-binding coverage.
• Docs: a file-sink section in the OpenTelemetry plugin configuration reference, including the non-goals above.

Start with OpenTelemetryConfig, the proposed TraceDestination and OtlpFileSinkSettings, then trace the existing SpanExporter and trace fan-out path. Review plugin validation and plugins edit, followed by the Python, Node.js, Go, and C FFI configuration surfaces. Done means both file formats, durability and path restrictions, destination validation, binding coverage, independent decoding tests, round-trip compatibility, and documentation meet the listed acceptance criteria.