WebSphere 9.0.5 & OTel Instrumentation Blockers

Describe the bug

It would be great to discuss few of the scenarios and opportunity to improvise instrumentation required for Traditional WebSphere (9.0.5) having 7 JAVA applications hosted over JVM and instrumentation done through OTel Java agent 2.18.1 having a local collector 0.144.0 which is connecting to again central collector gateway hosted in AKS (Cloud Environment).

I am working on WebSphere Traditional (9.0.5.15) Instrumentation through OTel 2.18.1 Zero instrumentation and metrics is only in scope . This WAS has 7 applications (JAVA 8) hosted over and the flow is WAS (OTel Attributes 7 Apps JVM Arguments) --otlp--> Local Collector (0.144.0) on same WAS host --otlp--> External Central collector (0.115.0) and I do see intermittent metrics and those are not consistent throughout and as well if any recycle or reboot happens it's completely changed.

Things are working fine for few applications but do not for all seven, have shared the OTel Attributes included in all 7 JVMs of one host/server.

The HTTP server metrics is not inconsistent across all applications in all nodes.

**JVM Arguments: **
-javaagent:/alt/dist/java/OTel/opentelemetry-javaagent-all/2.18.1/lib/opentelemetry-javaagent-2.18.1.jar
-Dotel.service.name=ALT-OTEL-QA-READ
-Dotel.metrics.exporter=otlp
-Dotel.traces.exporter=none
-Dotel.logs.exporter=none
-Dotel.exporter.otlp.protocol=http/protobuf
-Dotel.exporter.otlp.endpoint=http://127.0.0.1:4318/
-Dotel.instrumentation.common.default-enabled=true
-Dotel.semconv-stability.opt-in=http,database
-Dotel.instrumentation.runtime-telemetry.enabled=true
-Dotel.instrumentation.servlet.enabled=true
-Dotel.instrumentation.http.server.emit-experimental-telemetry=true
-Dotel.instrumentation.http.client.emit-experimental-telemetry=true
-Dotel.javaagent.debug=true
-Dotel.javaagent.logging=simple
-Dotel.otlp.export.interval=10000
-Dotel.instrumentation.jdbc.enabled=true
-Dotel.instrumentation.jdbc-datasource.enabled=true
-Dotel.instrumentation.kafka.enabled=true
-Dotel.instrumentation.messaging.experimental.receive-telemetry.enabled=true
-Dotel.instrumentation.apache-httpclient.enabled=true
-Dotel.instrumentation.okhttp.enabled=true
-Dotel.instrumentation.jaxrs-client.enabled=true
-Dotel.instrumentation.jaxws.enabled=true
-Dotel.resource.attributes=eonid=56778,env=qa,app=ALT-OTEL-QA-READ,service.instance.id=ALT-OTEL-QA-READ-aoq678190,was.server.name=aoq678190.qa.alt.com

Any clue on this would be appreciated if anything I am missing here to include in the JVM arguments or any recommendations for the same would be very helpful.

Steps to reproduce

Set up WAS 9.0.5.15 with multiple application (JVM) with Java 8 and try to instrument with OTel 2.8.1 and if you restart and all server metrics are not uniformed and it's keep changing with each JVM recycle .

Expected behavior

The Server metrics should be consistent across with or without restarts .

Actual behavior

The Server metrics is not consistent across all JVM/Applications with or without restarts.

Javaagent or library instrumentation version

Java agent 2.18.1

Environment

JDK:1.8 / JAVA 8.0 JRE 8.0
OS: RHEL 7

Additional context

No response

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

Start by reproducing the issue with WebSphere 9.0.5.15, Java 8, the 2.18.1 Java agent, and the local-to-central collector path described in the JVM arguments. Review the agent debug output and compare metrics across all seven applications before and after JVM recycling. Done means identifying why metrics differ and documenting a reliable configuration or reproduction-based fix.