open-telemetry / open-telemetry/opentelemetry-java-instrumentation
K8s OpenTelemetry Java Agent, missing metrics?
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 1.2k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 228
Description
Hi,
We are migrating our Wildfly VM's to Kubernetes. We used to use Prometheus to scrape metrics but we want to switch to OpenTelemetry. We are currently using the OpenTelemetry Operator with the Instrumentation CR:
apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
name: java-instrumentation
namespace: monitoring
spec:
exporter:
endpoint: http://alloy-receiver.kube-monitoring:4318
propagators:
- tracecontext
- baggage
sampler:
type: parentbased_traceidratio
argument: "1"
java:
image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-java:2.27.0
env:
- name: OTEL_EXPORTER_OTLP_PROTOCOL
value: http/protobuf
- name: OTEL_JMX_TARGET_SYSTEM
value: wildfly
- name: OTEL_METRICS_EXPORTER
value: otlp
This works, but we are missing some metrics that we hope to be able to get. For example, with Prometheus we get wildfly_undertow_active_sessions. Now with OTEL we getwildfly_active_sessions but this value stays empty. There should also be metrics for JDBC connections, especially one for max pool connections, but we do not seem to get them via OTEL. We are not sure what we are missing. If I understand correctly, by setting the OTEL_JMX_TARGET_SYSTEM to wildfly, we should already receive a lot of these metrics. I read about the JDBC metrics here (https://github.com/open-telemetry/semantic-conventions/blob/main/docs/db/database-metrics.md), but I am not sure if these include the max pool connections metric.
What do we need to set to get the JDBC (and undertow if it is indeed not the same) metric?
Sorry about the vague issue, I'm just not sure on how to continue. I also wasn't sure if I should set this here or in the opentelemetry-operator repo.
Thanks a lot!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with the Java agent's JMX instrumentation path and the OTEL_JMX_TARGET_SYSTEM=wildfly configuration; compare emitted wildfly_active_sessions with the expected Undertow and JDBC metrics. Reproduce using the supplied Instrumentation CR, then determine whether additional configuration is required or the metrics are unsupported. Done means the required settings or a concrete instrumentation gap is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kubernetes
- Domain
- backend, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100