feat(observability): OCSF event export from gateway to external SIEM destinations
@krishicks is already working on this.
Since Aug 20, 2026.
- Dominant language
- Rust
- Stars
- 8.7k
- Forks
- 1.3k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 253
Description
User Story
As a security or platform operator running OpenShell in production, I want the gateway to export OCSF security events to an external SIEM or log aggregator so that sandbox activity appears in my existing security operations workflows without a custom intermediary.
Problem Statement
OpenShell already relays OCSF events from the sandbox supervisor to the gateway via the PushSandboxLogs gRPC RPC. The gateway collects them in an in-memory TracingLogBus (a 2000-line ring buffer per sandbox) and makes them available through openshell logs and WatchSandbox. The relay chain ends there. No mechanism exists to export events from the gateway to an external destination such as a SIEM, log aggregator, or persistent store.
As a result, operators must build and maintain an external bridge that reads from openshell logs output and forwards it. A PoC confirmed this path works (parsing the shorthand text stream and forwarding to Splunk HEC), but it is a workaround: the bridge must stay running alongside the gateway, re-parses already-structured events from their text representation, and provides no backpressure or delivery guarantees.
Impact / Why This Matters
Without a native export surface, OCSF events are ephemeral. They exist only in the gateway's in-memory buffer for the lifetime of the sandbox and are not retained after bus.remove() is called. For security compliance workflows — including incident investigation, SAFE evidence preservation, and audit reporting — events that exist only in memory are not evidence.
The openshell logs workaround requires a persistent sidecar process with its own failure mode, adds an unnecessary parse-and-reserialize step on already-structured data, and cannot provide delivery guarantees or backpressure signaling. It also ties export behavior to the CLI interface rather than the gateway's operational configuration.
The gateway already holds all OCSF events from all sandboxes. It is the correct place to export them.
Proposed Design
Add a configurable OCSF export destination to the gateway. The minimal viable form is a structured log file or a configurable HTTP endpoint (Splunk HEC, OpenSearch, or a generic webhook):
[openshell.gateway.ocsf_export]
destination = "http://splunk-hec:8088/services/collector"
token_file = "/etc/openshell/hec-token"
format = "jsonl" # jsonl | splunk_hec
batch_size = 100
flush_ms = 500
The exporter should consume from TracingLogBus, not from openshell logs, to avoid re-parsing structured events. It should operate with backpressure (drop with counter, not block) consistent with the existing LogPushLayer semantics. On gateway restart, events already flushed need not be re-sent; the exporter need not guarantee exactly-once delivery.
For operators using OpenShift Logging or a Vector/Fluent Bit daemonset, a structured JSONL file destination (written by the gateway, picked up by the log collector) is an equally valid alternative and avoids the gateway needing outbound HTTP credentials.
The export surface is orthogonal to schema version selection (#2662) and trace correlation (#2640), but composes with both: exported events should carry trace_id/span_id when available, and should respect the configured ocsf_schema_version.
Acceptance Criteria
- OCSF events emitted by sandboxes on a configured gateway reach the configured external destination without a custom intermediary process.
- The exporter operates with backpressure: when the destination is unavailable, events are dropped with a counter rather than blocking the gateway.
- Exported events are structured OCSF JSON, not re-parsed shorthand text.
- The feature composes with #2640 (trace correlation fields) and #2662 (schema version selection).
- When no export destination is configured, behavior is unchanged.
Agent Investigation
crates/openshell-supervisor-process/src/log_push.rs—LogPushLayerpushes OCSF shorthand events from sandbox to gateway viaPushSandboxLogsgRPC RPC.crates/openshell-server/src/tracing_bus.rs—TracingLogBusis the gateway-side in-memory broadcast bus.bus.remove()drops all events on sandbox teardown.- Related: #1055 (Enterprise Observability umbrella), #1933 (audit/event log placeholder), #2507 (OTEL gateway export surface — explicitly excludes OCSF), #2640 (trace correlation fields), #2662 (OCSF schema version selection), #2745 (sandbox evidence bundle).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.