kestra-io / kestra-io/plugin-ai

Add OTEL trace capture via koltp for LLM requests

Open
#339 0 comments 0 reactions 0 assignees View on GitHub
area/analytics area/docs area/plugin
Dominant language
Java
Stars
9
Forks
24
Avg merge
1d 23h
Merged PRs (30d)
15

Description

## Context

[koltp](https://github.com/kestra-io/koltp) is a cross-platform process wrapper that captures OTEL traces, metrics (CPU/mem/network/IO), and structured logs from any process.

When a task runner has `monitoring.enabled: true`, koltp wraps the process and forwards OTEL data back to Kestra via `::{"oltp":}::` markers on stdout.

## Goal

For `plugin-ai`, koltp will automatically capture all HTTP requests emitted to LLM APIs (OpenAI, Anthropic, etc.) as OTEL traces — including prompt content and response metadata — without any code changes to the plugin itself, as long as the LLM client library uses standard HTTP.

## Work

- Validate that LLM client libraries used in `plugin-ai` emit OTEL traces or are instrumented via auto-instrumentation when an OTEL collector endpoint is available (set by koltp)
- Add documentation / task property notes explaining what traces are captured when `taskRunner.monitoring.enabled: true`
- If auto-instrumentation is insufficient, add explicit OpenTelemetry SDK calls to capture LLM request/response spans

## Value

- Visibility into every LLM call made by a flow: prompt, model, latency, token usage
- Stored inside Kestra's trace store, queryable over time
- Also forwarded to any external OTEL collector if configured

## Related

- koltp: https://github.com/kestra-io/koltp
- System usage tracking issue: https://github.com/kestra-io/kestra/issues/310
- Task runner monitoring prop (to be added per task runner)
- Implementation challenge: https://app.notion.com/p/kestra-io/Implementation-challenge-37c36907f7b580ea9af9c86569af20b3

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the LLM client libraries used in plugin-ai and the task runner monitoring property notes, then check koltp's OTEL endpoint and marker behavior. Validate whether automatic instrumentation captures the requested spans; if not, determine the scope of explicit SDK instrumentation. Done means traces are captured when taskRunner.monitoring.enabled is true and the behavior is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, documentation, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.