Partial span processor

What problem do you want to solve?

When you have very long running processes, there is no visibility until a span is ended. A process can run for hours and in the meantime you don't know whether it's still active or not. Additionally, in case of an ungraceful shutdown, the spans are never pushed to the collector, resulting in missing crucial data.

Describe the solution you'd like

The PartialSpanProcessor is an OpenTelemetry span processor that emits logs at regular intervals (referred to as "heartbeats") during the lifetime of a span and when the span ends. This processor is useful for monitoring long-running spans by providing periodic updates.

Features:

• Emits periodic heartbeat logs for active spans.
• Logs span data in JSON format.
• Configurable heartbeat interval, initial delay, and processing interval.
• Supports integration with any logging framework.

Describe alternatives you've considered

All alternatives were hacky workarounds.

Additional Context

There is a quite old issue on the otel spec regarding periodically exporting active spans in order to gain some sort of visibility for very long running processes.

https://github.com/open-telemetry/opentelemetry-specification/issues/373

There are many problems with exporting partial spans and the issue remains open.

We discussed the issue on the Spec community call, a couple months ago and the suggested solution was to periodically export logs or events. Also, the java-contrib has a package with processors that have been doing data conversions.

For more info, check this comment and the ones below it

https://github.com/open-telemetry/opentelemetry-specification/issues/373#issuecomment-2590963273

It was suggested that the individual sdk contrib repos are a good start to contribute such a processor so that people can start using it, experiment with it and provide feedback. If it turns out to be something useful then it might end up in the spec and the main repos.

At G-Research, we've also implemented a custom collector that acts as both a receiver and an exporter so that we can gather the logs, filter them and query the results.

We already have this implemented as a standalone extension of python sdk, but we'd like to contribute this back to a community. We're open to suggestions and further discussions regarding our PartialActivityProcessor.

Would you like to implement a fix?

Yes

Start by reviewing the linked OpenTelemetry specification issue and its referenced comments, then compare the proposed processor with the existing standalone Python SDK extension. No repository files or tests are named; done would require agreeing on the processor's design and integrating a community-supported implementation.