spring-cloud / spring-cloud/spring-cloud-stream

Kafka Reactive Binder does not work with multiple bindings

Open
#2,965 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement ideal-for-contribution
Dominant language
Java
Stars
1.1k
Forks
646
Avg merge
2d 3h
Merged PRs (30d)
8

Description

When using the Kafka Reactive Binder to consume messages from two different Kafka Cluster the app start fails with onfigReactiveWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context

I have a similar setup like the following:

spring:
  cloud:
    stream:
      binders:
        kafka1:
          type: kafka
          environment:
            spring:
              cloud:
                stream:
                  kafka:
                    binder:
                      brokers: kafka-cluster-1:9092
                      configuration:
                        security.protocol: SSL
                        ssl.truststore.location: /path/to/truststore.jks
                        ssl.truststore.password: yourpassword
        kafka2:
          type: kafka
          environment:
            spring:
              cloud:
                stream:
                  kafka:
                    binder:
                      brokers: kafka-cluster-2:9092
                      configuration:
                        security.protocol: SSL
                        ssl.truststore.location: /path/to/truststore.jks
                        ssl.truststore.password: yourpassword

      bindings:
        input1-in-0:
          destination: topic1
          binder: kafka1
        input2-in-0:
          destination: topic2
          binder: kafka2

and

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-binder-kafka-reactive</artifactId>
</dependency>

I tried to add the dependency:

  <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-stream-binder-kafka</artifactId>
    </dependency>

so that the configuration works but then the reactive part is ignored and it seems that the messages are consumed on a non reactive thread container-0-C-1 instead of nc-read-local-1 .

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the YAML configuration with two Kafka binders and the spring-cloud-stream-binder-kafka-reactive dependency, then compare it with the non-reactive Kafka binder setup described in the issue. Done means both clusters can be consumed through separate bindings while the reactive consumer remains in use and application startup succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.