newrelic / newrelic/newrelic-java-agent
Agent don't work with reactor-kafka (automatic or manual instrumentation)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 240
- Forks
- 170
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 21
Description
Description
I have app that works with spring boot and webflux and the kafka consumers using reactor-kafka, but any trace, transaction have sending to newrelic. I tried dowgrade all versions (spring boot, agent, webflux...) but it didn't work. All web transactions works fine (or maybe, http integrations or reactive mongodb is not tracing too) but the custom have no trace there,
Follow versions that I using:
webflux: 2.2.0.RELEASE
agent: 7.11.1
reactor-kafka: 1.3.14
reactor-netty: 0.9.25.RELEASE
For understand the problem, follow de code that configure my consumer, considering that my consumer implement a interface KafkaReactiveConsumer:
@Configuration
class ReactorKafkaConsumer(
private val receiver: ReceiverOptions<String, Any>,
private val tracingKafkaConsumerFactory: TracingKafkaConsumerFactory,
private val consumers: List<KafkaReactiveConsumer>
) {
private val log = KotlinLogging.logger { }
private val disposables = Disposables.composite()
@PostConstruct
fun connect() {
consumers.map { consumer ->
disposables.add(
consumes(consumer).subscribe()
)
}
}
@PreDestroy
fun disconnect() {
this.disposables.dispose()
}
fun consumes(consumer: KafkaReactiveConsumer): Flux<Void> {
val receiverSubscribed = receiver.subscription(consumer.getTopics())
.addAssignListener { partitions -> log.debug("on partitions assigned {}", partitions) }
.addRevokeListener { partitions -> log.debug("on partitions revoked {}", partitions) }
return KafkaReceiver.create(tracingKafkaConsumerFactory, receiverSubscribed).receive()
.flatMap {
consumer.consumer(it)
}
}
}
Sample of my consumer...
@Trace(dispatcher = true, metricName = "consumer-document-reference-capture")
override fun consumer(receive: ReceiverRecord<String, Any>): Mono<Void> {
return receive.toMono()
.doOnNext { log.info { "Received: ${it.value()}" } }
.flatMap { message ->
val offset = message.receiverOffset()
message.toMono()
.map {
objectMapper.readValue<Object>(it.value() as String)
}
.filterWhen { event -> callService(event) } }
.zipWhen {
callOtherService(it)
}
.map {
Tuples.of(callOneOtherServiceit.t1.payload), it.t2)
}
.map { callServiceFromThisClass(it) }
.flatMap {
callMoreOneService(it)
.thenReturn(message)
}
.flatMap {
val event = objectMapper.readValue<Object>(it.value() as String).event
callOneLastService.create(event.uuid)
}
.doOnSuccess {
log.debug { "Success to consuming message with id ${receive.key()} ack on message" }
offset.acknowledge()
}
.onErrorResume { t ->
log.error(t) { "Consuming messages failed" }
offset.acknowledge()
Mono.empty()
}
}
}
And I've got the from agent:
2022-11-22T22:35:10,036-0300 [99365 85] com.newrelic FINEST: Setting transaction name to "consumer-document-reference-capture" for transaction com.newrelic.agent.Transaction@37ff828f using LEGACY scheme
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINER: Added log attribute "timestamp": 1.669.167.310.035
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINER: Added log attribute "logger.fqcn": mu.internal.LocationAwareKLogger
2022-11-22T22:35:10,036-0300 [99365 85] com.newrelic FINEST: tracerFinished: com.newrelic.agent.tracers.OtherRootTracer@17ad6e12 opcode: 176 in transactionActivity com.newrelic.agent.TransactionActivity@0
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINER: Added log attribute "logger.name": com.unico.auto.order.application.usecase.SaveDocumentReference
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINER: Added log attribute "thread.name": kafka-producer-network-thread | auto-order-local
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINER: Added log attribute "level": INFO
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINEST: Added event of type LogEvent
2022-11-22T22:35:10,036-0300 [99365 85] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@37ff828f: Activity com.newrelic.agent.TransactionActivity@0 finished with opcode 176
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,036-0300 [99365 85] com.newrelic FINEST: Tracer Attributes for com.newrelic.agent.tracers.OtherRootTracer@17ad6e12 are {async_context=reactive-kafka-auto-order-local-3, code.namespace=com.samples.test.MyConsumer, code.function=consumer}
2022-11-22T22:35:10,036-0300 [99365 85] com.newrelic FINEST: tx response time set: 308.443
2022-11-22T22:35:10,036-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,036-0300 [99365 85] com.newrelic FINER: Transaction OtherTransaction/Custom/consumer-document-reference-capture for request: consumer-document-reference-capture finished 0ms com.newrelic.agent.Transaction@37ff828f
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,037-0300 [99365 85] com.newrelic FINEST: SlowQueryAggregator: addSlowQueriesFromTransaction: no listener
2022-11-22T22:35:10,037-0300 [99365 209] com.newrelic FINER: Transaction com.newrelic.agent.Transaction@6676b6b1: ignoring link call because there is no started txa to link to: null.
2022-11-22T22:35:10,038-0300 [99365 209] com.newrelic FINEST: Expiring token com.newrelic.agent.TokenImpl@32531767 on transaction com.newrelic.agent.Transaction@6676b6b1
2022-11-22T22:35:10,038-0300 [99365 209] com.newrelic FINEST: Transaction com.newrelic.agent.Transaction@6676b6b1: expired token com.newrelic.agent.TokenImpl@32531767
2022-11-22T22:35:10,038-0300 [99365 85] com.newrelic FINER: Added log attribute "thread.id": 85
2022-11-22T22:35:10,038-0300 [99365 209] com.newrelic FINER: Token has already been expired com.newrelic.agent.TokenImpl@32531767.
2022-11-22T22:35:10,038-0300 [99365 85] com.newrelic FINER: Added log attribute "timestamp": 1.669.167.310.038
2022-11-22T22:35:10,039-0300 [99365 85] com.newrelic FINER: Added log attribute "logger.fqcn": mu.internal.LocationAwareKLogger
2022-11-22T22:35:10,039-0300 [99365 85] com.newrelic FINER: Added log attribute "logger.name": com.unico.auto.order.adapter.kafka.in.DocumentReferenceConsumer
2022-11-22T22:35:10,039-0300 [99365 85] com.newrelic FINER: Added log attribute "thread.name": reactive-kafka-auto-order-local-3
2022-11-22T22:35:10,039-0300 [99365 85] com.newrelic FINER: Added log attribute "level": INFO
2022-11-22T22:35:10,039-0300 [99365 85] com.newrelic FINEST: Added event of type LogEvent
2022-11-22T22:35:10,039-0300 [99365 209] com.newrelic FINER: Transaction null: ignoring link call because transaction already on thread.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository files or tests are identified. Start by reproducing the supplied Reactor Kafka consumer with the listed Spring Boot, agent, Reactor Kafka, and Reactor Netty versions, then compare the agent logs with automatic and manual instrumentation. Done means the consumer's trace and transaction are reported to New Relic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kotlin, spring-boot
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100