Kafka SASL_PLAINTEXT problem
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 4k
- Forks
- 818
- Avg merge
- 1h 14m
- Merged PRs (30d)
- 1
Description
Hi,
I am trying to configure Burrow with Kafka SASL. I am using kafka image from wurstmeister with the following configuration:
```
kafka:
image: wurstmeister/kafka:1.0.0
network_mode: host
restart: on-failure
environment:
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: SASL_PLAINTEXT:SASL_PLAINTEXT,OUTSIDE:PLAINTEXT
KAFKA_ADVERTISED_PROTOCOL_NAME: OUTSIDE
KAFKA_PROTOCOL_NAME: SASL_PLAINTEXT
KAFKA_ADVERTISED_PORT: "9094"
KAFKA_ZOOKEEPER_CONNECT: "183.155.2.197:2181"
KAFKA_INTER_BROKER_PROTOCOL: PLAIN
KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.auth.SimpleAclAuthorizer
KAFKA_SUPER_USERS: "User:admin"
KAFKA_OPTS: "-Djava.security.auth.login.config=jaas.conf"
```
jaas.conf:
```
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="admin"
password="adminpass"
};
```
And Burrow with the following configuration:
```
[general]
access-control-allow-origin="*"
[logging]
level="debug"
[zookeeper]
servers=["183.155.2.197:2181"]
[client-profile.kafka-profile]
kafka-version="1.0.0"
client-id="burrow-client"
sasl="mysasl"
[sasl.mysasl]
username="admin"
password="adminpass"
[cluster.my_cluster]
class-name="kafka"
client-profile="kafka-profile"
servers=["183.155.2.197:9094"]
topic-refresh=120
offset-refresh=10
[consumer.consumer_kafka]
class-name="kafka"
cluster="my_cluster"
servers=["183.155.2.197:9094"]
client-profile="kafka-profile"
start-latest=false
offsets-topic="__consumer_offsets"
group-blacklist="^(console-consumer-|python-kafka-consumer-).*$"
[consumer.consumer_zk]
class-name="kafka_zk"
cluster="my_cluster"
servers=["183.155.2.197:2181"]
zookeeper-timeout=30
group-blacklist="^(console-consumer-|python-kafka-consumer-).*$"
[httpserver.default]
address=":8002"
```
When i turn off SASL_PLAINTEXT on Kafka, Burrow starts with success and is connected to kafka brokers. With this configuration i get the following error during burrow<->kafka connection:
```
2018-01-25 11:17:57,887] ERROR Closing socket for 183.155.2.197:9094-183.155.2.197:51406-107 because of error (kafka.network.Processor)
org.apache.kafka.common.errors.InvalidRequestException: Error parsing request header. Our best guess of the apiKey is: 97
Caused by: org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'client_id': Error reading string of length 25709, only 9 bytes available
at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:77)
at org.apache.kafka.common.requests.RequestHeader.parse(RequestHeader.java:121)
at kafka.network.Processor.$anonfun$processCompletedReceives$1(SocketServer.scala:549)
at kafka.network.Processor.$anonfun$processCompletedReceives$1$adapted(SocketServer.scala:545)
at scala.collection.Iterator.foreach(Iterator.scala:929)
at scala.collection.Iterator.foreach$(Iterator.scala:929)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
at scala.collection.IterableLike.foreach(IterableLike.scala:71)
at scala.collection.IterableLike.foreach$(IterableLike.scala:70)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at kafka.network.Processor.processCompletedReceives(SocketServer.scala:545)
at kafka.network.Processor.run(SocketServer.scala:453)
at java.lang.Thread.run(Thread.java:748)
[2018-01-25 11:17:57,887] ERROR Exception while processing request from 183.155.2.197:9094-183.155.2.197:51406-107 (kafka.network.Processor)
org.apache.kafka.common.errors.InvalidRequestException: Error parsing request header. Our best guess of the apiKey is: 97
Caused by: org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'client_id': Error reading string of length 25709, only 9 bytes available
at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:77)
at org.apache.kafka.common.requests.RequestHeader.parse(RequestHeader.java:121)
at kafka.network.Processor.$anonfun$processCompletedReceives$1(SocketServer.scala:549)
at kafka.network.Processor.$anonfun$processCompletedReceives$1$adapted(SocketServer.scala:545)
at scala.collection.Iterator.foreach(Iterator.scala:929)
at scala.collection.Iterator.foreach$(Iterator.scala:929)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
at scala.collection.IterableLike.foreach(IterableLike.scala:71)
at scala.collection.IterableLike.foreach$(IterableLike.scala:70)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at kafka.network.Processor.processCompletedReceives(SocketServer.scala:545)
at kafka.network.Processor.run(SocketServer.scala:453)
at java.lang.Thread.run(Thread.java:748)
[2018-01-25 11:20:37,060] INFO [GroupMetadataManager brokerId=1001] Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file or test is identified in the report. Start by reproducing the Burrow-to-Kafka connection with the supplied Docker, JAAS, and Burrow configurations, then trace the connection using the reported broker error. Done means identifying the SASL_PLAINTEXT compatibility or configuration cause and documenting a verified working setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go, kafka
- Domain
- distributed-systems, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100